Update commons-lang exclude rule to exclude it everywhere by Swiddis · Pull Request #3932 · opensearch-project/sql

Swiddis · 2025-07-28T21:05:55Z

Description

commons-lang was excluded as part of the calcite-core import for the core config, but still was being pulled in via another transient dependency chain in ppl. This PR updates the exclude to remove commons-lang everywhere.

Related Issues

N/A

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

* Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> (cherry picked from commit edb3a0d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) * Update commons-lang exclude rule to exclude it everywhere * Undo removal in core --------- (cherry picked from commit edb3a0d) Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com>

…cite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

…cite) (opensearch-project#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (opensearch-project#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (opensearch-project#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (opensearch-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (opensearch-project#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (opensearch-project#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (opensearch-project#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of opensearch-project#3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (opensearch-project#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (opensearch-project#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (opensearch-project#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (opensearch-project#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (opensearch-project#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (opensearch-project#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (opensearch-project#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (opensearch-project#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (opensearch-project#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (opensearch-project#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (opensearch-project#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (opensearch-project#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (opensearch-project#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (opensearch-project#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (opensearch-project#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (opensearch-project#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (opensearch-project#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (opensearch-project#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (opensearch-project#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (opensearch-project#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (opensearch-project#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0)

…ection Features (Cal… (#4102) * `fields` Command Enhancement - Advanced Field Selection Features (Calcite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0) * Fixing compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing JDK compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fixing java 11 compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Swiddis · 2025-07-28T21:05:55Z

Description

commons-lang was excluded as part of the calcite-core import for the core config, but still was being pulled in via another transient dependency chain in ppl. This PR updates the exclude to remove commons-lang everywhere.

Related Issues

N/A

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

* Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> (cherry picked from commit edb3a0d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) * Update commons-lang exclude rule to exclude it everywhere * Undo removal in core --------- (cherry picked from commit edb3a0d) Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com>

…cite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

…cite) (opensearch-project#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (opensearch-project#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (opensearch-project#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (opensearch-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (opensearch-project#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (opensearch-project#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (opensearch-project#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of opensearch-project#3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (opensearch-project#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (opensearch-project#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (opensearch-project#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (opensearch-project#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (opensearch-project#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (opensearch-project#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (opensearch-project#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (opensearch-project#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (opensearch-project#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (opensearch-project#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (opensearch-project#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (opensearch-project#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (opensearch-project#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (opensearch-project#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (opensearch-project#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (opensearch-project#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (opensearch-project#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (opensearch-project#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (opensearch-project#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (opensearch-project#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (opensearch-project#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0)

…ection Features (Cal… (#4102) * `fields` Command Enhancement - Advanced Field Selection Features (Calcite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0) * Fixing compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing JDK compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fixing java 11 compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Swiddis · 2025-07-28T21:05:55Z

Description

commons-lang was excluded as part of the calcite-core import for the core config, but still was being pulled in via another transient dependency chain in ppl. This PR updates the exclude to remove commons-lang everywhere.

Related Issues

N/A

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

* Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> (cherry picked from commit edb3a0d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) * Update commons-lang exclude rule to exclude it everywhere * Undo removal in core --------- (cherry picked from commit edb3a0d) Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com>

…cite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

…cite) (opensearch-project#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (opensearch-project#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (opensearch-project#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (opensearch-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (opensearch-project#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (opensearch-project#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (opensearch-project#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of opensearch-project#3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (opensearch-project#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (opensearch-project#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (opensearch-project#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (opensearch-project#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (opensearch-project#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (opensearch-project#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (opensearch-project#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (opensearch-project#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (opensearch-project#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (opensearch-project#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (opensearch-project#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (opensearch-project#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (opensearch-project#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (opensearch-project#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (opensearch-project#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (opensearch-project#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (opensearch-project#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (opensearch-project#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (opensearch-project#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (opensearch-project#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (opensearch-project#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0)

…ection Features (Cal… (#4102) * `fields` Command Enhancement - Advanced Field Selection Features (Calcite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0) * Fixing compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing JDK compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fixing java 11 compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Swiddis · 2025-07-28T21:05:55Z

Description

commons-lang was excluded as part of the calcite-core import for the core config, but still was being pulled in via another transient dependency chain in ppl. This PR updates the exclude to remove commons-lang everywhere.

Related Issues

N/A

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

* Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> (cherry picked from commit edb3a0d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) * Update commons-lang exclude rule to exclude it everywhere * Undo removal in core --------- (cherry picked from commit edb3a0d) Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com>

…cite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

…cite) (opensearch-project#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (opensearch-project#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (opensearch-project#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (opensearch-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (opensearch-project#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (opensearch-project#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (opensearch-project#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of opensearch-project#3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (opensearch-project#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (opensearch-project#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (opensearch-project#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (opensearch-project#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (opensearch-project#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (opensearch-project#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (opensearch-project#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (opensearch-project#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (opensearch-project#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (opensearch-project#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (opensearch-project#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (opensearch-project#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (opensearch-project#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (opensearch-project#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (opensearch-project#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (opensearch-project#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (opensearch-project#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (opensearch-project#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (opensearch-project#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (opensearch-project#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (opensearch-project#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0)

…ection Features (Cal… (#4102) * `fields` Command Enhancement - Advanced Field Selection Features (Calcite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0) * Fixing compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing JDK compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fixing java 11 compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Swiddis · 2025-07-28T21:05:55Z

Description

commons-lang was excluded as part of the calcite-core import for the core config, but still was being pulled in via another transient dependency chain in ppl. This PR updates the exclude to remove commons-lang everywhere.

Related Issues

N/A

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

* Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> (cherry picked from commit edb3a0d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) * Update commons-lang exclude rule to exclude it everywhere * Undo removal in core --------- (cherry picked from commit edb3a0d) Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com>

…cite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

…cite) (opensearch-project#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (opensearch-project#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (opensearch-project#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (opensearch-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (opensearch-project#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (opensearch-project#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (opensearch-project#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of opensearch-project#3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (opensearch-project#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (opensearch-project#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (opensearch-project#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (opensearch-project#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (opensearch-project#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (opensearch-project#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (opensearch-project#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (opensearch-project#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (opensearch-project#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (opensearch-project#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (opensearch-project#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (opensearch-project#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (opensearch-project#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (opensearch-project#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (opensearch-project#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (opensearch-project#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (opensearch-project#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (opensearch-project#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (opensearch-project#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (opensearch-project#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (opensearch-project#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0)

…ection Features (Cal… (#4102) * `fields` Command Enhancement - Advanced Field Selection Features (Calcite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0) * Fixing compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing JDK compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fixing java 11 compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Swiddis · 2025-07-28T21:05:55Z

Description

commons-lang was excluded as part of the calcite-core import for the core config, but still was being pulled in via another transient dependency chain in ppl. This PR updates the exclude to remove commons-lang everywhere.

Related Issues

N/A

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

* Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> (cherry picked from commit edb3a0d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) * Update commons-lang exclude rule to exclude it everywhere * Undo removal in core --------- (cherry picked from commit edb3a0d) Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com>

…cite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

…cite) (opensearch-project#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (opensearch-project#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (opensearch-project#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (opensearch-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (opensearch-project#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (opensearch-project#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (opensearch-project#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of opensearch-project#3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (opensearch-project#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (opensearch-project#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (opensearch-project#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (opensearch-project#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (opensearch-project#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (opensearch-project#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (opensearch-project#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (opensearch-project#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (opensearch-project#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (opensearch-project#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (opensearch-project#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (opensearch-project#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (opensearch-project#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (opensearch-project#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (opensearch-project#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (opensearch-project#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (opensearch-project#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (opensearch-project#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (opensearch-project#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (opensearch-project#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (opensearch-project#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0)

…ection Features (Cal… (#4102) * `fields` Command Enhancement - Advanced Field Selection Features (Calcite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0) * Fixing compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing JDK compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fixing java 11 compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Swiddis · 2025-07-28T21:05:55Z

Description

commons-lang was excluded as part of the calcite-core import for the core config, but still was being pulled in via another transient dependency chain in ppl. This PR updates the exclude to remove commons-lang everywhere.

Related Issues

N/A

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

* Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> (cherry picked from commit edb3a0d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) * Update commons-lang exclude rule to exclude it everywhere * Undo removal in core --------- (cherry picked from commit edb3a0d) Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com>

…cite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

…cite) (opensearch-project#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (opensearch-project#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (opensearch-project#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (opensearch-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (opensearch-project#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (opensearch-project#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (opensearch-project#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of opensearch-project#3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (opensearch-project#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (opensearch-project#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (opensearch-project#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (opensearch-project#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (opensearch-project#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (opensearch-project#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (opensearch-project#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (opensearch-project#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (opensearch-project#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (opensearch-project#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (opensearch-project#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (opensearch-project#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (opensearch-project#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (opensearch-project#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (opensearch-project#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (opensearch-project#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (opensearch-project#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (opensearch-project#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (opensearch-project#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (opensearch-project#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (opensearch-project#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0)

…ection Features (Cal… (#4102) * `fields` Command Enhancement - Advanced Field Selection Features (Calcite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0) * Fixing compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing JDK compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fixing java 11 compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Swiddis · 2025-07-28T21:05:55Z

Description

commons-lang was excluded as part of the calcite-core import for the core config, but still was being pulled in via another transient dependency chain in ppl. This PR updates the exclude to remove commons-lang everywhere.

Related Issues

N/A

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

* Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> (cherry picked from commit edb3a0d) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) * Update commons-lang exclude rule to exclude it everywhere * Undo removal in core --------- (cherry picked from commit edb3a0d) Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com>

…cite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

…cite) (opensearch-project#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (opensearch-project#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (opensearch-project#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (opensearch-project#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (opensearch-project#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (opensearch-project#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (opensearch-project#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (opensearch-project#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of opensearch-project#3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (opensearch-project#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (opensearch-project#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (opensearch-project#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (opensearch-project#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (opensearch-project#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (opensearch-project#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (opensearch-project#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (opensearch-project#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (opensearch-project#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (opensearch-project#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (opensearch-project#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (opensearch-project#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (opensearch-project#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (opensearch-project#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (opensearch-project#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (opensearch-project#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (opensearch-project#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (opensearch-project#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (opensearch-project#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (opensearch-project#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (opensearch-project#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (opensearch-project#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (opensearch-project#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (opensearch-project#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (opensearch-project#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0)

…ection Features (Cal… (#4102) * `fields` Command Enhancement - Advanced Field Selection Features (Calcite) (#3970) * Add support for space-separated fields in addition to comma-separated Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Byte number should treated as Long in doc values (#3928) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding table as alias of the fields command Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix create PIT permissions issue (#3921) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Convert like function call to wildcard query for Calcite filter pushdown (#3915) * Convert like function call to wildcard query for Calcite filter pushdown Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix V2 expression like function bug and match its behavior in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix like default escape in Calcite Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix tests Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix spotless check Signed-off-by: Songkan Tang <songkant@amazon.com> * Address comments Signed-off-by: Songkan Tang <songkant@amazon.com> * Fix SQL IT correctness Signed-off-by: Songkan Tang <songkant@amazon.com> * Remove test log Signed-off-by: Songkan Tang <songkant@amazon.com> * Minor improve one CalciteLikeQueryIT Signed-off-by: Songkan Tang <songkant@amazon.com> --------- Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update commons-lang exclude rule to exclude it everywhere (#3932) * Update commons-lang exclude rule to exclude it everywhere Signed-off-by: Simeon Widdis <sawiddis@amazon.com> * Undo removal in core Signed-off-by: Simeon Widdis <sawiddis@amazon.com> --------- Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding wildcard support to fields command in Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Mixed delimiter support - Support both space and comma delimiters in the same command for table and fields Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding widlcard support to non-Calcite engine and updating documentation with new features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support function argument coercion with Calcite (#3914) * Change the use of SqlTypeFamily.STRING to SqlTypeFamily.CHARACTER as the string family contains binary, which is not expected for most functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement basic argument type coercion at RelNode level Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Conform type checkers with their definition in documentation - string as an input is removed if it is not in the document - string as an input is kept if it is in the document, even if it can be implicitly cast - use PPLOperandTypes as much as possible Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Implement type widening for comparator functions - Add COMPARATORS set to BuiltinFunctionName for identifying comparison operators - Implement widenArguments method in CoercionUtils to find widest compatible type - Apply type widening to comparator functions before applying type casting - Add detailed JavaDoc to explain coercion methods Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update error messages of datetime functions with invalid args Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify datetime-string compare logic with implict coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor resolve with coercion Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Move down argument cast for reduce function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Merge comparators and their IP variants so that coercion works for IP comparison - when not merging, ip comparing will also pass the type checker of Calcite's comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor ip comparator to comparator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Refactor ip comparator to comparator" This reverts commit c539056. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert "Merge comparators and their IP variants so that coercion works for IP comparison" This reverts commit bd9f3bb. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Rule out ip from built-in comparator via its type checker Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Restrict CompareIP's parameter type Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Revert to previous implementation of CompareIpFunction to temporarily fix ip comparison pushdown problems (udt not correctly serialized; ip comparison is not converted to range query) Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test argument coercion explain Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix error msg in CalcitePPLFunctionTypeTest Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing command in index.rst (#3943) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Append limit operator for QUEERY_SIZE_LIMIT (#3940) * Append limit operator for QUEERY_SIZE_LIMIT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add LogicalSystemLimit Signed-off-by: Heng Qian <qianheng@amazon.com> * Revert part of #3880 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT after merging main Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code cleaning and fixing tests Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Performing code maintenance and adding more test cases Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Doing some code cleaning and maintenance Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing code and implementation logic Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add issue template specific for PPL commands and queries (#3962) * Add issue template specific for PPL commands and queries Signed-off-by: Anas Alkouz <aalkouz@amazon.com> * Add section for Dataset/schema information, add reminders for the customers to remove any sensitive datas Signed-off-by: Anas Alkouz <aalkouz@amazon.com> --------- Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increase the precision of sum return type (#3974) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disable a failed PPL query fallback to v2 by default (#3952) * Disable a failed PPL query fallback to v2 by default Signed-off-by: Lantao Jin <ltjin@amazon.com> * Workaround the permissionIT Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Update the maven snapshot publish endpoint and credential (#3806) Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add release notes for 3.2.0 (#3985) Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing documentation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Taking care of comments left by Tomo Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding full wildcard support functionality Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Increment version to 3.2.0-SNAPSHOT (#3819) Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support `reverse` command with Calcite (#3867) * Implement reverse Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse integ tests and unit tests Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse documentation Signed-off-by: Selina Song <selsong@amazon.com> * Modify reverse test and documentation Signed-off-by: Selina Song <selsong@amazon.com> * Fix limit pushdown bug when reverse comes before head * Revert "Fix limit pushdown bug when reverse comes before head" This reverts commit 087c936. Signed-off-by: Selina Song <selsong@amazon.com> * Fix grammar, naming, and test cases. Pushdown reverted will be in 2nd PR. Signed-off-by: Selina Song <selsong@amazon.com> * Fix reverse tests: update logical plans, format with Spotless - Updated expected logical plans and Spark SQL in reverse tests - Applied Spotless to fix formatting Signed-off-by: Selina Song <selsong@amazon.com> * Fix OS version in build 3.1.0 Signed-off-by: Selina Song <selsong@amazon.com> * Add note on limitation to rst Signed-off-by: Selina Song <selsong@amazon.com> * Move explain IT to correct file, add Anonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add reverse to index.rst Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Co-authored-by: Selina Song <selsong@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Pass JOIN_TIME_OUT value to keepalive (#3826) * Fix JOIN_TIME_OUT Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix style Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix log-rethrow Signed-off-by: Kai Huang <ahkcs@amazon.com> * Restructure Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix format Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove unused PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove reflection, add hintConfig Signed-off-by: Kai Huang <ahkcs@amazon.com> * Added Unit test, restructured to use existing methods Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * add IT tests Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * format fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * trim IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * formatting Signed-off-by: Kai Huang <ahkcs@amazon.com> * Update legacy/src/main/java/org/opensearch/sql/legacy/query/planner/physical/node/pointInTime/PointInTime.java Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * update .gitignore Signed-off-by: Kai Huang <ahkcs@amazon.com> * deletion Signed-off-by: Kai Huang <ahkcs@amazon.com> * restore main PIT Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Changed seenFields to Hashset instead of LinkedHashSet Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Creating a rule only for the fields/table commands to avoid any interference Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * This commit is in response of PR comments left by Tomo and Chen Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing Integration test failure Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding anonymizer tests, wildcard unit tests, etc Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling Calcite for enhance fields features Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Disabling automatic de-deduplication when Calcite is disabled Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding cross-cluster IT test Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Adding a dedicated Cross-cluster IT test file for Calcite Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing formatting issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Improving widlcard logic and exception message Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Addressing comments left by Tomo regarding wildcard logic implementation Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add missing udfs in v3 (#3957) * add math udfs Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix decimal bug Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * make general udf adapter Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add math IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add rst Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * fix error Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * change signum IT Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> * add javadoc Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> --------- Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fix snapshot uploading (#4006) * fix snapshot uploading Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add comment Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix DOUBLE to STRING cast rendering zero values in scientific notation (#3982) * Fix casting double 0.0 to string Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Fix float to string casting precision lost with custom FormatNumberFunction This commit fixes float to string casting by replacing the use of SqlLibraryOperators.FORMAT_NUMBER with a custom FormatNumberFunction implementation. The new implementation converts the number to a BigDecimal before formatting to preserve precision and avoid issues like 6.2 becoming 6.199999809265137. Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the implementation of fp number to string cast Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Update implementation of NumberToStringFunction Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Cast decimal with NUMBER_TO_STRING function Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test cast decimal Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Eliminate reliance on assert in Calcite for integration test (#4016) * Move num-of-column check of in subquery ahead from RexSubQuery.java#L78 because assert is disabled in production Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Increase script.context.filter.max_compilations_rate for SQLCorrectnessIT Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Check script.disable_max_compilations_rate before setting context-specific compilations rate Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor: remove some methods in tests to upper level to reduce duplication Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Replace plugin-level setting strings with private test-specific ones Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Prevent aggregation push down when it has inner filter (#4002) * Prevent aggregation push down when it has inner filter Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT & Remove log Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix 4009 Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix span on negative timestamp (#4017) * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix span on negative timestamp Signed-off-by: Heng Qian <qianheng@amazon.com> * typo Signed-off-by: Heng Qian <qianheng@amazon.com> * Refine code Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Skip script encoding when run explain with 'extended' (#3930) * No need to decode script when run explain command Signed-off-by: Lantao Jin <ltjin@amazon.com> * address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * Do not encoding when explain format is 'extended' Signed-off-by: Lantao Jin <ltjin@amazon.com> * Rename the thread local var Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT after merge main Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Implement type checking for aggregation functions with Calcite (#4024) * Remove getTypeChecker from FunctionImp interface Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor registerExternalFunction to registerExternalOperator Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Do not register GEOIP function if got incompatible client Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Create scaffold for type checking of aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Add type checkers for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test type checking for aggregation functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Allow equal expression as a function argument (#4001) * Remove named function arg from functions other than table functions Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Test eval if function with equal as condition Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Push down IP comparison as range query with Calcite (#3959) * Add reverse op for compare ip to support pushdown Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Pushdown ip comparison Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Refactor CompareIpFunction to use SqlKind directly Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> * Simplify the overriding of reverse() for IP comparators Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> --------- Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * eval sum, avg implementation (#3986) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix PPL eval command string concatenation with + operator (#4020) * eval command support Signed-off-by: Kai Huang <ahkcs@amazon.com> * improvment Signed-off-by: Kai Huang <ahkcs@amazon.com> * Refactor Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix CI Signed-off-by: Kai Huang <ahkcs@amazon.com> * fixes Signed-off-by: Kai Huang <ahkcs@amazon.com> * fix Signed-off-by: Kai Huang <ahkcs@amazon.com> * Add IT Signed-off-by: Kai Huang <ahkcs@amazon.com> * remove redundant tests Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support script push down on text field (#4010) * Support script push down on text field Signed-off-by: Heng Qian <qianheng@amazon.com> * Fix IT Signed-off-by: Heng Qian <qianheng@amazon.com> * Add UT for struct type push down Signed-off-by: Heng Qian <qianheng@amazon.com> --------- Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Enhance sort command in PPL (#3934) * enhance sort command Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add integ tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update documentation Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update default and tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update analyzer test Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update reverse sort direction Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update docs Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add javadoc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix failing tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update integ tests for query size limit change Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add explainit for desc and type cast Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add tests for desc Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix formatting Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * make count optional Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * add cross cluster tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * fix tests Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * normalize count in AST node Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * default null count to 0 Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> * update logicalsort default constructor Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> --------- Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Add example for String concat in eval.rst (#4075) * Add example for String concat in eval.rst Signed-off-by: Kai Huang <ahkcs@amazon.com> * mention calcite enabling Signed-off-by: Kai Huang <ahkcs@amazon.com> --------- Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Support pushdown dedup with Calcite (#3972) * Support pushdown dedup with Calcite Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix IT Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comments Signed-off-by: Lantao Jin <ltjin@amazon.com> * Fix flaky test Signed-off-by: Lantao Jin <ltjin@amazon.com> * Address comment Signed-off-by: Lantao Jin <ltjin@amazon.com> * delete useless codes Signed-off-by: Lantao Jin <ltjin@amazon.com> * Add more ITs Signed-off-by: Lantao Jin <ltjin@amazon.com> --------- Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fix CI failure because of plan having changed (#4077) Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Empty commit Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com> (cherry picked from commit f16f1c0) * Fixing compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * Fixing JDK compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> * fixing java 11 compatibility issues Signed-off-by: Aaron Alvarez <aaarone@amazon.com> --------- Signed-off-by: Aaron Alvarez <aaarone@amazon.com> Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Songkan Tang <songkant@amazon.com> Signed-off-by: Simeon Widdis <sawiddis@amazon.com> Signed-off-by: Yuanchun Shen <yuanchu@amazon.com> Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: Heng Qian <qianheng@amazon.com> Signed-off-by: Anas Alkouz <aalkouz@amazon.com> Signed-off-by: opensearch-ci <opensearch-infra@amazon.com> Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Kai Huang <ahkcs@amazon.com> Signed-off-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Signed-off-by: Xinyu Hao <haoxinyu@amazon.com> Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com> Signed-off-by: Aaron Alvarez <900908alvarezaaron@gmail.com> Co-authored-by: Aaron Alvarez <aaarone@amazon.com> Co-authored-by: Lantao Jin <ltjin@amazon.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com> Co-authored-by: Songkan Tang <songkant@amazon.com> Co-authored-by: Simeon Widdis <sawiddis@gmail.com> Co-authored-by: Yuanchun Shen <yuanchu@amazon.com> Co-authored-by: Peng Huo <penghuo@gmail.com> Co-authored-by: qianheng <qianheng@amazon.com> Co-authored-by: Anas Alkouz <aalkouz@amazon.com> Co-authored-by: Zelin Hao <zelinhao@amazon.com> Co-authored-by: Sayali Gaikawad <gaiksaya@amazon.com> Co-authored-by: opensearch-ci <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com> Co-authored-by: Kai Huang <105710027+ahkcs@users.noreply.github.com> Co-authored-by: Louis Chu <lingzhichu.clz@gmail.com> Co-authored-by: Xinyu Hao <75524174+ishaoxy@users.noreply.github.com> Co-authored-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Swiddis added 2 commits July 28, 2025 21:03

Update commons-lang exclude rule to exclude it everywhere
fce082d
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Undo removal in core
f818c7e
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Swiddis added the maintenance Improves code quality, but not the product label Jul 28, 2025

Swiddis marked this pull request as ready for review July 28, 2025 21:37

Swiddis requested review from GumpacG, LantaoJin, MaxKsyunz, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, noCharger, penghuo, ps48, qianheng-aws, seankao-az and ykmr1224 as code owners July 28, 2025 21:37

Swiddis enabled auto-merge (squash) July 28, 2025 21:37

RyanL1997 approved these changes Jul 28, 2025
View reviewed changes

LantaoJin approved these changes Jul 29, 2025
View reviewed changes

LantaoJin added the backport 2.19-dev label Jul 31, 2025

qianheng-aws approved these changes Jul 31, 2025
View reviewed changes

Swiddis merged commit edb3a0d into opensearch-project:mainJul 31, 2025
27 checks passed

opensearch-trigger-botBot mentioned this pull request Jul 31, 2025
[Backport 2.19-dev] Update commons-lang exclude rule to exclude it everywhere #3956
Merged

Swiddis added 2 commits July 28, 2025 21:03

Update commons-lang exclude rule to exclude it everywhere
fce082d
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Undo removal in core
f818c7e
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Swiddis added the maintenance Improves code quality, but not the product label Jul 28, 2025

Swiddis marked this pull request as ready for review July 28, 2025 21:37

Swiddis requested review from GumpacG, LantaoJin, MaxKsyunz, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, noCharger, penghuo, ps48, qianheng-aws, seankao-az and ykmr1224 as code owners July 28, 2025 21:37

Swiddis enabled auto-merge (squash) July 28, 2025 21:37

RyanL1997 approved these changes Jul 28, 2025
View reviewed changes

LantaoJin approved these changes Jul 29, 2025
View reviewed changes

LantaoJin added the backport 2.19-dev label Jul 31, 2025

qianheng-aws approved these changes Jul 31, 2025
View reviewed changes

Swiddis merged commit edb3a0d into opensearch-project:mainJul 31, 2025
27 checks passed

opensearch-trigger-botBot mentioned this pull request Jul 31, 2025
[Backport 2.19-dev] Update commons-lang exclude rule to exclude it everywhere #3956
Merged

Swiddis added 2 commits July 28, 2025 21:03

Update commons-lang exclude rule to exclude it everywhere
fce082d
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Undo removal in core
f818c7e
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Swiddis added the maintenance Improves code quality, but not the product label Jul 28, 2025

Swiddis marked this pull request as ready for review July 28, 2025 21:37

Swiddis requested review from GumpacG, LantaoJin, MaxKsyunz, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, noCharger, penghuo, ps48, qianheng-aws, seankao-az and ykmr1224 as code owners July 28, 2025 21:37

Swiddis enabled auto-merge (squash) July 28, 2025 21:37

RyanL1997 approved these changes Jul 28, 2025
View reviewed changes

LantaoJin approved these changes Jul 29, 2025
View reviewed changes

LantaoJin added the backport 2.19-dev label Jul 31, 2025

qianheng-aws approved these changes Jul 31, 2025
View reviewed changes

Swiddis merged commit edb3a0d into opensearch-project:mainJul 31, 2025
27 checks passed

opensearch-trigger-botBot mentioned this pull request Jul 31, 2025
[Backport 2.19-dev] Update commons-lang exclude rule to exclude it everywhere #3956
Merged

Swiddis added 2 commits July 28, 2025 21:03

Update commons-lang exclude rule to exclude it everywhere
fce082d
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Undo removal in core
f818c7e
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Swiddis added the maintenance Improves code quality, but not the product label Jul 28, 2025

Swiddis marked this pull request as ready for review July 28, 2025 21:37

Swiddis requested review from GumpacG, LantaoJin, MaxKsyunz, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, noCharger, penghuo, ps48, qianheng-aws, seankao-az and ykmr1224 as code owners July 28, 2025 21:37

Swiddis enabled auto-merge (squash) July 28, 2025 21:37

RyanL1997 approved these changes Jul 28, 2025
View reviewed changes

LantaoJin approved these changes Jul 29, 2025
View reviewed changes

LantaoJin added the backport 2.19-dev label Jul 31, 2025

qianheng-aws approved these changes Jul 31, 2025
View reviewed changes

Swiddis merged commit edb3a0d into opensearch-project:mainJul 31, 2025
27 checks passed

opensearch-trigger-botBot mentioned this pull request Jul 31, 2025
[Backport 2.19-dev] Update commons-lang exclude rule to exclude it everywhere #3956
Merged

Swiddis added 2 commits July 28, 2025 21:03

Update commons-lang exclude rule to exclude it everywhere
fce082d
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Undo removal in core
f818c7e
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Swiddis added the maintenance Improves code quality, but not the product label Jul 28, 2025

Swiddis marked this pull request as ready for review July 28, 2025 21:37

Swiddis requested review from GumpacG, LantaoJin, MaxKsyunz, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, noCharger, penghuo, ps48, qianheng-aws, seankao-az and ykmr1224 as code owners July 28, 2025 21:37

Swiddis enabled auto-merge (squash) July 28, 2025 21:37

RyanL1997 approved these changes Jul 28, 2025
View reviewed changes

LantaoJin approved these changes Jul 29, 2025
View reviewed changes

LantaoJin added the backport 2.19-dev label Jul 31, 2025

qianheng-aws approved these changes Jul 31, 2025
View reviewed changes

Swiddis merged commit edb3a0d into opensearch-project:mainJul 31, 2025
27 checks passed

opensearch-trigger-botBot mentioned this pull request Jul 31, 2025
[Backport 2.19-dev] Update commons-lang exclude rule to exclude it everywhere #3956
Merged

Swiddis added 2 commits July 28, 2025 21:03

Update commons-lang exclude rule to exclude it everywhere
fce082d
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Undo removal in core
f818c7e
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Swiddis added the maintenance Improves code quality, but not the product label Jul 28, 2025

Swiddis marked this pull request as ready for review July 28, 2025 21:37

Swiddis requested review from GumpacG, LantaoJin, MaxKsyunz, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, noCharger, penghuo, ps48, qianheng-aws, seankao-az and ykmr1224 as code owners July 28, 2025 21:37

Swiddis enabled auto-merge (squash) July 28, 2025 21:37

RyanL1997 approved these changes Jul 28, 2025
View reviewed changes

LantaoJin approved these changes Jul 29, 2025
View reviewed changes

LantaoJin added the backport 2.19-dev label Jul 31, 2025

qianheng-aws approved these changes Jul 31, 2025
View reviewed changes

Swiddis merged commit edb3a0d into opensearch-project:mainJul 31, 2025
27 checks passed

opensearch-trigger-botBot mentioned this pull request Jul 31, 2025
[Backport 2.19-dev] Update commons-lang exclude rule to exclude it everywhere #3956
Merged

Swiddis added 2 commits July 28, 2025 21:03

Update commons-lang exclude rule to exclude it everywhere
fce082d
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Undo removal in core
f818c7e
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Swiddis added the maintenance Improves code quality, but not the product label Jul 28, 2025

Swiddis marked this pull request as ready for review July 28, 2025 21:37

Swiddis requested review from GumpacG, LantaoJin, MaxKsyunz, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, noCharger, penghuo, ps48, qianheng-aws, seankao-az and ykmr1224 as code owners July 28, 2025 21:37

Swiddis enabled auto-merge (squash) July 28, 2025 21:37

RyanL1997 approved these changes Jul 28, 2025
View reviewed changes

LantaoJin approved these changes Jul 29, 2025
View reviewed changes

LantaoJin added the backport 2.19-dev label Jul 31, 2025

qianheng-aws approved these changes Jul 31, 2025
View reviewed changes

Swiddis merged commit edb3a0d into opensearch-project:mainJul 31, 2025
27 checks passed

opensearch-trigger-botBot mentioned this pull request Jul 31, 2025
[Backport 2.19-dev] Update commons-lang exclude rule to exclude it everywhere #3956
Merged

Swiddis added 2 commits July 28, 2025 21:03

Update commons-lang exclude rule to exclude it everywhere
fce082d
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Undo removal in core
f818c7e
Signed-off-by: Simeon Widdis <sawiddis@amazon.com>

Swiddis added the maintenance Improves code quality, but not the product label Jul 28, 2025

Swiddis marked this pull request as ready for review July 28, 2025 21:37

Swiddis requested review from GumpacG, LantaoJin, MaxKsyunz, YANG-DB, Yury-Fridlyand, acarbonetto, anirudha, dai-chen, derek-ho, forestmvey, joshuali925, kavithacm, mengweieric, noCharger, penghuo, ps48, qianheng-aws, seankao-az and ykmr1224 as code owners July 28, 2025 21:37

Swiddis enabled auto-merge (squash) July 28, 2025 21:37

RyanL1997 approved these changes Jul 28, 2025
View reviewed changes

LantaoJin approved these changes Jul 29, 2025
View reviewed changes

LantaoJin added the backport 2.19-dev label Jul 31, 2025

qianheng-aws approved these changes Jul 31, 2025
View reviewed changes

Swiddis merged commit edb3a0d into opensearch-project:mainJul 31, 2025
27 checks passed

opensearch-trigger-botBot mentioned this pull request Jul 31, 2025
[Backport 2.19-dev] Update commons-lang exclude rule to exclude it everywhere #3956
Merged

Uh oh!

Conversation

Swiddis commented Jul 28, 2025

Description

Related Issues

Check List

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Swiddis commented Jul 28, 2025

Description

Related Issues

Check List

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Swiddis commented Jul 28, 2025

Description

Related Issues

Check List

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Swiddis commented Jul 28, 2025

Description

Related Issues

Check List

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Swiddis commented Jul 28, 2025

Description

Related Issues

Check List

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants