Skip to content

fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) - #5732

Open
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom
ahkcs:fix/dedup-object-scalar-mapping-5685
Open

fix: gracefully handle scalar values under object-typed fields when decoding results (#5685)#5732
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom
ahkcs:fix/dedup-object-scalar-mapping-5685

Conversation

@ahkcs

@ahkcsahkcs commented Aug 31, 2026•
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

On the Calcite path, PPL dedup pushes down as a composite aggregation whose surviving row is fetched via top_hits and decoded from the nested _source. When a query spans a wildcard/alias over indices that disagree on whether a path is an object or a scalar, the cross-index mapping merge types the path as an object (STRUCT) while some documents store a scalar there. Decoding that scalar as a struct threw and failed the whole query:

java.sql.SQLException: ... class java.lang.String cannot be cast to class java.util.Map

This guards the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a scalar-under-object value to null, completing the graceful handling #5618 added for the geo_point and scalar branches (it did not cover the object/STRUCT branch). The catch is scoped to ClassCastException — this branch is also the whole-document parse entry point and each nested object level is wrapped independently, so a broader catch would swallow legitimate errors raised deeper in the recursion.

Issues Resolved

Closes#5685

Behavior (before → after)

ScenarioBeforeAfter
dedup over wildcard, object-vs-scalar conflict on an object path500 String cannot be cast to Mapsucceeds; conflicting field → null
dedup over wildcard, scalar-vs-object conflict (mirror)already handled by #5618unchanged (regression-guarded)
Unsupported-type parse, deeply nestedIllegalStateException (correct)unchanged (not masked)

Testing

  • Unit: OpenSearchExprValueFactoryTest.constructStructWithScalarValueReturnsNull
  • YAML REST: rest-api-spec/test/issues/5685.yml — object-vs-scalar conflict and its mirror (scalar-vs-object).
  • Full OpenSearchExprValueFactoryTest and the :opensearch unit suite pass; integ-test compiles.

Check List

  • New functionality includes testing.
  • Commits are signed per the DCO using --signoff.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions

github-actionsBot commented Aug 31, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

(Review updated until commit 3f0f955)

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions

github-actionsBot commented Aug 31, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

Latest suggestions up to 3f0f955

Explore these optional code suggestions:

CategorySuggestion Impact
General
Add logging for caught exceptions

Consider logging the ClassCastException at debug or trace level before returning
null. This would help diagnose mapping conflicts in production environments while
maintaining the graceful degradation behavior.

opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]

 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value found under object-typed field '{}', returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5

__

Why: While logging the ClassCastException could aid debugging, the suggestion doesn't address a critical issue. The PR already handles the exception gracefully, and adding logging is a minor enhancement for observability rather than a functional improvement.

Low

Previous suggestions

Suggestions up to commit 22f7212
CategorySuggestion Impact
General
Log caught ClassCastException for debugging

Catching ClassCastException silently may hide legitimate bugs in parseStruct or its
callees. Consider logging the exception with context (field name, content type) to
aid debugging while still returning null for the known scalar-under-object scenario.

opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]

 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value encountered under object-typed field '{}': {}", field, e.getMessage());
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5

__

Why: Adding logging for the caught ClassCastException would help with debugging and understanding when this edge case occurs. However, the comment in the code already explains this is a specific fix for issue #5685 (scalar under object-typed field), and the catch is intentionally scoped to avoid hiding other errors. The suggestion is valid but offers moderate improvement since the code already has explanatory comments.

Low
Suggestions up to commit 7d340e8
CategorySuggestion Impact
General
Add logging for mapping conflicts

Consider logging the ClassCastException at debug or trace level before returning
null. This would help diagnose mapping conflicts in production without failing
queries, providing visibility into when scalar-to-struct coercion occurs.

opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [234-238]

 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Field '{}' mapped as object but contains scalar value, returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5

__

Why: While logging the ClassCastException would provide useful diagnostic information for mapping conflicts, this is a minor enhancement rather than a critical fix. The suggestion is valid and would improve observability, but the current implementation already handles the error gracefully by returning null.

Low

@ahkcs
ahkcsforce-pushed the fix/dedup-object-scalar-mapping-5685 branch from 7d340e8 to 22f7212CompareAugust 31, 2026 22:06
@github-actions

github-actionsBot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 22f7212

RyanL1997
RyanL1997 reviewed Aug 31, 2026
View reviewed changes
Comment thread...earch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java Outdated
|| type == STRUCT) {
return parseStruct(content, field, supportArrays);
// A scalar under an object-typed field (wildcard over indices with conflicting mappings)
// would throw here; return null instead (#5685). CCE-scoped: this is also the whole-document

@RyanL1997RyanL1997Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets just remove the inline PR link - instead if it is necessary lets to the yaml test with issue id.

@ahkcsahkcsAug 31, 2026

Copy link
Copy Markdown
CollaboratorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use yaml test as IT

@ahkcs
Fix dedup 500 on cross-index object/scalar mapping conflict (opensear…
3f0f955
…ch-project#5685)
On the Calcite path, `dedup` pushes down as a composite aggregation whose
surviving row is fetched via top_hits and decoded from the nested _source.
When a wildcard spans indices that disagree on whether a path is an object
or a scalar, the merged mapping types the path as an object (STRUCT) while
some documents store a scalar there. Decoding that scalar as a struct threw
`java.sql.SQLException: ... class java.lang.String cannot be cast to
class java.util.Map`, failing the whole query (100% on the reporting
customer's `source=logs-pr172502-*`).
Guard the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a
scalar-under-object value to null, completing the graceful handling opensearch-project#5618
added for the geo_point and scalar branches. The catch is scoped to
ClassCastException because this branch is also the whole-document parse
entry point and each nested object level is wrapped independently, so a
broader catch would swallow legitimate errors raised deeper in the recursion.
Adds a factory unit regression and a yamlRestTest (issues/5685.yml) covering
the object-vs-scalar conflict and its mirror (scalar-vs-object, guarding opensearch-project#5618).
Signed-off-by: Kai Huang <ahkcs@amazon.com>
@ahkcs
ahkcsforce-pushed the fix/dedup-object-scalar-mapping-5685 branch from 22f7212 to 3f0f955CompareAugust 31, 2026 22:40
@github-actions

github-actionsBot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 3f0f955

@dai-chen

dai-chen commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Related to #5610

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@RyanL1997RyanL1997RyanL1997 left review comments
@ps48ps48Awaiting requested review from ps48ps48 is a code owner
@joshuali925joshuali925Awaiting requested review from joshuali925joshuali925 is a code owner
@dai-chendai-chenAwaiting requested review from dai-chendai-chen is a code owner
@mengweiericmengweiericAwaiting requested review from mengweiericmengweieric is a code owner
@vamsimanoharvamsimanoharAwaiting requested review from vamsimanoharvamsimanohar is a code owner
@SwiddisSwiddisAwaiting requested review from SwiddisSwiddis is a code owner
@penghuopenghuoAwaiting requested review from penghuopenghuo is a code owner
@anirudhaanirudhaAwaiting requested review from anirudhaanirudha is a code owner
@acarbonettoacarbonettoAwaiting requested review from acarbonettoacarbonetto is a code owner
@ykmr1224ykmr1224Awaiting requested review from ykmr1224ykmr1224 is a code owner
@LantaoJinLantaoJinAwaiting requested review from LantaoJinLantaoJin is a code owner
@noChargernoChargerAwaiting requested review from noChargernoCharger is a code owner
@qianheng-awsqianheng-awsAwaiting requested review from qianheng-awsqianheng-aws is a code owner
@yuancuyuancuAwaiting requested review from yuancuyuancu is a code owner
@songkant-awssongkant-awsAwaiting requested review from songkant-awssongkant-aws is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

bugFix

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[BUG] dedup on a dotted object-hierarchy field throws "String cannot be cast to java.util.Map" (Calcite path)

3 participants

@ahkcs@dai-chen@RyanL1997
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) by ahkcs · Pull Request #5732 · opensearch-project/sql · GitHub
Skip to content
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) - #5732
Open
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Open
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685)#5732
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Conversation
@ahkcs
@ahkcsahkcs
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
On the Calcite path, PPL dedup pushes down as a composite aggregation whose surviving row is fetched via top_hits and decoded from the nested _source. When a query spans a wildcard/alias over indices that disagree on whether a path is an object or a scalar, the cross-index mapping merge types the path as an object (STRUCT) while some documents store a scalar there. Decoding that scalar as a struct threw and failed the whole query:
java.sql.SQLException: ... class java.lang.String cannot be cast to class java.util.Map

This guards the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a scalar-under-object value to null, completing the graceful handling #5618 added for the geo_point and scalar branches (it did not cover the object/STRUCT branch). The catch is scoped to ClassCastException — this branch is also the whole-document parse entry point and each nested object level is wrapped independently, so a broader catch would swallow legitimate errors raised deeper in the recursion.
Issues Resolved
Closes#5685
Behavior (before → after)
ScenarioBeforeAfter
dedup over wildcard, object-vs-scalar conflict on an object path500 String cannot be cast to Mapsucceeds; conflicting field → null
dedup over wildcard, scalar-vs-object conflict (mirror)already handled by #5618unchanged (regression-guarded)
Unsupported-type parse, deeply nestedIllegalStateException (correct)unchanged (not masked)
Testing
  • Unit: OpenSearchExprValueFactoryTest.constructStructWithScalarValueReturnsNull
  • YAML REST: rest-api-spec/test/issues/5685.yml — object-vs-scalar conflict and its mirror (scalar-vs-object).
  • Full OpenSearchExprValueFactoryTest and the :opensearch unit suite pass; integ-test compiles.
Check List
  •  New functionality includes testing.
  •  Commits are signed per the DCO using --signoff.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Reviewer Guide 🔍
(Review updated until commit 3f0f955)
Here are some key observations to aid the review process:
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Code Suggestions ✨
Latest suggestions up to 3f0f955
Explore these optional code suggestions:
CategorySuggestion Impact
General
Add logging for caught exceptions
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production environments while 
maintaining the graceful degradation behavior.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value found under object-typed field '{}', returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException could aid debugging, the suggestion doesn't address a critical issue. The PR already handles the exception gracefully, and adding logging is a minor enhancement for observability rather than a functional improvement.
Low

Previous suggestions
Suggestions up to commit 22f7212
CategorySuggestion Impact
General
Log caught ClassCastException for debugging
Catching ClassCastException silently may hide legitimate bugs in parseStruct or its 
callees. Consider logging the exception with context (field name, content type) to 
aid debugging while still returning null for the known scalar-under-object scenario.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value encountered under object-typed field '{}': {}", field, e.getMessage());
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: Adding logging for the caught ClassCastException would help with debugging and understanding when this edge case occurs. However, the comment in the code already explains this is a specific fix for issue #5685 (scalar under object-typed field), and the catch is intentionally scoped to avoid hiding other errors. The suggestion is valid but offers moderate improvement since the code already has explanatory comments.
Low

Suggestions up to commit 7d340e8
CategorySuggestion Impact
General
Add logging for mapping conflicts
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production without failing 
queries, providing visibility into when scalar-to-struct coercion occurs.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [234-238]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Field '{}' mapped as object but contains scalar value, returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException would provide useful diagnostic information for mapping conflicts, this is a minor enhancement rather than a critical fix. The suggestion is valid and would improve observability, but the current implementation already handles the error gracefully by returning null.
Low

@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
7d340e8 to
22f7212CompareAugust 31, 2026 22:06
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 22f7212
RyanL1997
RyanL1997
reviewed
Aug 31, 2026
View reviewed changes
Comment thread...earch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java
Outdated

 || type == STRUCT) {
 return parseStruct(content, field, supportArrays);
 // A scalar under an object-typed field (wildcard over indices with conflicting mappings)
 // would throw here; return null instead (#5685). CCE-scoped: this is also the whole-document
@RyanL1997RyanL1997Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets just remove the inline PR link - instead if it is necessary lets to the yaml test with issue id.
@ahkcsahkcsAug 31, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use yaml test as IT
@ahkcs
Fix dedup 500 on cross-index object/scalar mapping conflict (opensear…
3f0f955
…ch-project#5685)
On the Calcite path, `dedup` pushes down as a composite aggregation whose
surviving row is fetched via top_hits and decoded from the nested _source.
When a wildcard spans indices that disagree on whether a path is an object
or a scalar, the merged mapping types the path as an object (STRUCT) while
some documents store a scalar there. Decoding that scalar as a struct threw
`java.sql.SQLException: ... class java.lang.String cannot be cast to
class java.util.Map`, failing the whole query (100% on the reporting
customer's `source=logs-pr172502-*`).
Guard the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a
scalar-under-object value to null, completing the graceful handling opensearch-project#5618
added for the geo_point and scalar branches. The catch is scoped to
ClassCastException because this branch is also the whole-document parse
entry point and each nested object level is wrapped independently, so a
broader catch would swallow legitimate errors raised deeper in the recursion.
Adds a factory unit regression and a yamlRestTest (issues/5685.yml) covering
the object-vs-scalar conflict and its mirror (scalar-vs-object, guarding opensearch-project#5618).
Signed-off-by: Kai Huang <ahkcs@amazon.com>
@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
22f7212 to
3f0f955CompareAugust 31, 2026 22:40
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 3f0f955
@dai-chen
dai-chen
commented
Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Related to #5610
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@RyanL1997RyanL1997RyanL1997 left review comments
@ps48ps48Awaiting requested review from ps48ps48 is a code owner
@joshuali925joshuali925Awaiting requested review from joshuali925joshuali925 is a code owner
@dai-chendai-chenAwaiting requested review from dai-chendai-chen is a code owner
@mengweiericmengweiericAwaiting requested review from mengweiericmengweieric is a code owner
@vamsimanoharvamsimanoharAwaiting requested review from vamsimanoharvamsimanohar is a code owner
@SwiddisSwiddisAwaiting requested review from SwiddisSwiddis is a code owner
@penghuopenghuoAwaiting requested review from penghuopenghuo is a code owner
@anirudhaanirudhaAwaiting requested review from anirudhaanirudha is a code owner
@acarbonettoacarbonettoAwaiting requested review from acarbonettoacarbonetto is a code owner
@ykmr1224ykmr1224Awaiting requested review from ykmr1224ykmr1224 is a code owner
@LantaoJinLantaoJinAwaiting requested review from LantaoJinLantaoJin is a code owner
@noChargernoChargerAwaiting requested review from noChargernoCharger is a code owner
@qianheng-awsqianheng-awsAwaiting requested review from qianheng-awsqianheng-aws is a code owner
@yuancuyuancuAwaiting requested review from yuancuyuancu is a code owner
@songkant-awssongkant-awsAwaiting requested review from songkant-awssongkant-aws is a code owner

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels

bugFix

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[BUG] dedup on a dotted object-hierarchy field throws "String cannot be cast to java.util.Map" (Calcite path)


3 participants

@ahkcs@dai-chen@RyanL1997

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) by ahkcs · Pull Request #5732 · opensearch-project/sql · GitHub
Skip to content
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) - #5732
Open
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Open
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685)#5732
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Conversation
@ahkcs
@ahkcsahkcs
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
On the Calcite path, PPL dedup pushes down as a composite aggregation whose surviving row is fetched via top_hits and decoded from the nested _source. When a query spans a wildcard/alias over indices that disagree on whether a path is an object or a scalar, the cross-index mapping merge types the path as an object (STRUCT) while some documents store a scalar there. Decoding that scalar as a struct threw and failed the whole query:
java.sql.SQLException: ... class java.lang.String cannot be cast to class java.util.Map

This guards the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a scalar-under-object value to null, completing the graceful handling #5618 added for the geo_point and scalar branches (it did not cover the object/STRUCT branch). The catch is scoped to ClassCastException — this branch is also the whole-document parse entry point and each nested object level is wrapped independently, so a broader catch would swallow legitimate errors raised deeper in the recursion.
Issues Resolved
Closes#5685
Behavior (before → after)
ScenarioBeforeAfter
dedup over wildcard, object-vs-scalar conflict on an object path500 String cannot be cast to Mapsucceeds; conflicting field → null
dedup over wildcard, scalar-vs-object conflict (mirror)already handled by #5618unchanged (regression-guarded)
Unsupported-type parse, deeply nestedIllegalStateException (correct)unchanged (not masked)
Testing
  • Unit: OpenSearchExprValueFactoryTest.constructStructWithScalarValueReturnsNull
  • YAML REST: rest-api-spec/test/issues/5685.yml — object-vs-scalar conflict and its mirror (scalar-vs-object).
  • Full OpenSearchExprValueFactoryTest and the :opensearch unit suite pass; integ-test compiles.
Check List
  •  New functionality includes testing.
  •  Commits are signed per the DCO using --signoff.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Reviewer Guide 🔍
(Review updated until commit 3f0f955)
Here are some key observations to aid the review process:
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Code Suggestions ✨
Latest suggestions up to 3f0f955
Explore these optional code suggestions:
CategorySuggestion Impact
General
Add logging for caught exceptions
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production environments while 
maintaining the graceful degradation behavior.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value found under object-typed field '{}', returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException could aid debugging, the suggestion doesn't address a critical issue. The PR already handles the exception gracefully, and adding logging is a minor enhancement for observability rather than a functional improvement.
Low

Previous suggestions
Suggestions up to commit 22f7212
CategorySuggestion Impact
General
Log caught ClassCastException for debugging
Catching ClassCastException silently may hide legitimate bugs in parseStruct or its 
callees. Consider logging the exception with context (field name, content type) to 
aid debugging while still returning null for the known scalar-under-object scenario.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value encountered under object-typed field '{}': {}", field, e.getMessage());
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: Adding logging for the caught ClassCastException would help with debugging and understanding when this edge case occurs. However, the comment in the code already explains this is a specific fix for issue #5685 (scalar under object-typed field), and the catch is intentionally scoped to avoid hiding other errors. The suggestion is valid but offers moderate improvement since the code already has explanatory comments.
Low

Suggestions up to commit 7d340e8
CategorySuggestion Impact
General
Add logging for mapping conflicts
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production without failing 
queries, providing visibility into when scalar-to-struct coercion occurs.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [234-238]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Field '{}' mapped as object but contains scalar value, returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException would provide useful diagnostic information for mapping conflicts, this is a minor enhancement rather than a critical fix. The suggestion is valid and would improve observability, but the current implementation already handles the error gracefully by returning null.
Low

@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
7d340e8 to
22f7212CompareAugust 31, 2026 22:06
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 22f7212
RyanL1997
RyanL1997
reviewed
Aug 31, 2026
View reviewed changes
Comment thread...earch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java
Outdated

 || type == STRUCT) {
 return parseStruct(content, field, supportArrays);
 // A scalar under an object-typed field (wildcard over indices with conflicting mappings)
 // would throw here; return null instead (#5685). CCE-scoped: this is also the whole-document
@RyanL1997RyanL1997Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets just remove the inline PR link - instead if it is necessary lets to the yaml test with issue id.
@ahkcsahkcsAug 31, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use yaml test as IT
@ahkcs
Fix dedup 500 on cross-index object/scalar mapping conflict (opensear…
3f0f955
…ch-project#5685)
On the Calcite path, `dedup` pushes down as a composite aggregation whose
surviving row is fetched via top_hits and decoded from the nested _source.
When a wildcard spans indices that disagree on whether a path is an object
or a scalar, the merged mapping types the path as an object (STRUCT) while
some documents store a scalar there. Decoding that scalar as a struct threw
`java.sql.SQLException: ... class java.lang.String cannot be cast to
class java.util.Map`, failing the whole query (100% on the reporting
customer's `source=logs-pr172502-*`).
Guard the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a
scalar-under-object value to null, completing the graceful handling opensearch-project#5618
added for the geo_point and scalar branches. The catch is scoped to
ClassCastException because this branch is also the whole-document parse
entry point and each nested object level is wrapped independently, so a
broader catch would swallow legitimate errors raised deeper in the recursion.
Adds a factory unit regression and a yamlRestTest (issues/5685.yml) covering
the object-vs-scalar conflict and its mirror (scalar-vs-object, guarding opensearch-project#5618).
Signed-off-by: Kai Huang <ahkcs@amazon.com>
@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
22f7212 to
3f0f955CompareAugust 31, 2026 22:40
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 3f0f955
@dai-chen
dai-chen
commented
Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Related to #5610
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@RyanL1997RyanL1997RyanL1997 left review comments
@ps48ps48Awaiting requested review from ps48ps48 is a code owner
@joshuali925joshuali925Awaiting requested review from joshuali925joshuali925 is a code owner
@dai-chendai-chenAwaiting requested review from dai-chendai-chen is a code owner
@mengweiericmengweiericAwaiting requested review from mengweiericmengweieric is a code owner
@vamsimanoharvamsimanoharAwaiting requested review from vamsimanoharvamsimanohar is a code owner
@SwiddisSwiddisAwaiting requested review from SwiddisSwiddis is a code owner
@penghuopenghuoAwaiting requested review from penghuopenghuo is a code owner
@anirudhaanirudhaAwaiting requested review from anirudhaanirudha is a code owner
@acarbonettoacarbonettoAwaiting requested review from acarbonettoacarbonetto is a code owner
@ykmr1224ykmr1224Awaiting requested review from ykmr1224ykmr1224 is a code owner
@LantaoJinLantaoJinAwaiting requested review from LantaoJinLantaoJin is a code owner
@noChargernoChargerAwaiting requested review from noChargernoCharger is a code owner
@qianheng-awsqianheng-awsAwaiting requested review from qianheng-awsqianheng-aws is a code owner
@yuancuyuancuAwaiting requested review from yuancuyuancu is a code owner
@songkant-awssongkant-awsAwaiting requested review from songkant-awssongkant-aws is a code owner

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels

bugFix

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[BUG] dedup on a dotted object-hierarchy field throws "String cannot be cast to java.util.Map" (Calcite path)


3 participants

@ahkcs@dai-chen@RyanL1997

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) by ahkcs · Pull Request #5732 · opensearch-project/sql · GitHub
Skip to content
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) - #5732
Open
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Open
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685)#5732
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Conversation
@ahkcs
@ahkcsahkcs
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
On the Calcite path, PPL dedup pushes down as a composite aggregation whose surviving row is fetched via top_hits and decoded from the nested _source. When a query spans a wildcard/alias over indices that disagree on whether a path is an object or a scalar, the cross-index mapping merge types the path as an object (STRUCT) while some documents store a scalar there. Decoding that scalar as a struct threw and failed the whole query:
java.sql.SQLException: ... class java.lang.String cannot be cast to class java.util.Map

This guards the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a scalar-under-object value to null, completing the graceful handling #5618 added for the geo_point and scalar branches (it did not cover the object/STRUCT branch). The catch is scoped to ClassCastException — this branch is also the whole-document parse entry point and each nested object level is wrapped independently, so a broader catch would swallow legitimate errors raised deeper in the recursion.
Issues Resolved
Closes#5685
Behavior (before → after)
ScenarioBeforeAfter
dedup over wildcard, object-vs-scalar conflict on an object path500 String cannot be cast to Mapsucceeds; conflicting field → null
dedup over wildcard, scalar-vs-object conflict (mirror)already handled by #5618unchanged (regression-guarded)
Unsupported-type parse, deeply nestedIllegalStateException (correct)unchanged (not masked)
Testing
  • Unit: OpenSearchExprValueFactoryTest.constructStructWithScalarValueReturnsNull
  • YAML REST: rest-api-spec/test/issues/5685.yml — object-vs-scalar conflict and its mirror (scalar-vs-object).
  • Full OpenSearchExprValueFactoryTest and the :opensearch unit suite pass; integ-test compiles.
Check List
  •  New functionality includes testing.
  •  Commits are signed per the DCO using --signoff.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Reviewer Guide 🔍
(Review updated until commit 3f0f955)
Here are some key observations to aid the review process:
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Code Suggestions ✨
Latest suggestions up to 3f0f955
Explore these optional code suggestions:
CategorySuggestion Impact
General
Add logging for caught exceptions
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production environments while 
maintaining the graceful degradation behavior.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value found under object-typed field '{}', returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException could aid debugging, the suggestion doesn't address a critical issue. The PR already handles the exception gracefully, and adding logging is a minor enhancement for observability rather than a functional improvement.
Low

Previous suggestions
Suggestions up to commit 22f7212
CategorySuggestion Impact
General
Log caught ClassCastException for debugging
Catching ClassCastException silently may hide legitimate bugs in parseStruct or its 
callees. Consider logging the exception with context (field name, content type) to 
aid debugging while still returning null for the known scalar-under-object scenario.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value encountered under object-typed field '{}': {}", field, e.getMessage());
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: Adding logging for the caught ClassCastException would help with debugging and understanding when this edge case occurs. However, the comment in the code already explains this is a specific fix for issue #5685 (scalar under object-typed field), and the catch is intentionally scoped to avoid hiding other errors. The suggestion is valid but offers moderate improvement since the code already has explanatory comments.
Low

Suggestions up to commit 7d340e8
CategorySuggestion Impact
General
Add logging for mapping conflicts
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production without failing 
queries, providing visibility into when scalar-to-struct coercion occurs.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [234-238]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Field '{}' mapped as object but contains scalar value, returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException would provide useful diagnostic information for mapping conflicts, this is a minor enhancement rather than a critical fix. The suggestion is valid and would improve observability, but the current implementation already handles the error gracefully by returning null.
Low

@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
7d340e8 to
22f7212CompareAugust 31, 2026 22:06
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 22f7212
RyanL1997
RyanL1997
reviewed
Aug 31, 2026
View reviewed changes
Comment thread...earch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java
Outdated

 || type == STRUCT) {
 return parseStruct(content, field, supportArrays);
 // A scalar under an object-typed field (wildcard over indices with conflicting mappings)
 // would throw here; return null instead (#5685). CCE-scoped: this is also the whole-document
@RyanL1997RyanL1997Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets just remove the inline PR link - instead if it is necessary lets to the yaml test with issue id.
@ahkcsahkcsAug 31, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use yaml test as IT
@ahkcs
Fix dedup 500 on cross-index object/scalar mapping conflict (opensear…
3f0f955
…ch-project#5685)
On the Calcite path, `dedup` pushes down as a composite aggregation whose
surviving row is fetched via top_hits and decoded from the nested _source.
When a wildcard spans indices that disagree on whether a path is an object
or a scalar, the merged mapping types the path as an object (STRUCT) while
some documents store a scalar there. Decoding that scalar as a struct threw
`java.sql.SQLException: ... class java.lang.String cannot be cast to
class java.util.Map`, failing the whole query (100% on the reporting
customer's `source=logs-pr172502-*`).
Guard the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a
scalar-under-object value to null, completing the graceful handling opensearch-project#5618
added for the geo_point and scalar branches. The catch is scoped to
ClassCastException because this branch is also the whole-document parse
entry point and each nested object level is wrapped independently, so a
broader catch would swallow legitimate errors raised deeper in the recursion.
Adds a factory unit regression and a yamlRestTest (issues/5685.yml) covering
the object-vs-scalar conflict and its mirror (scalar-vs-object, guarding opensearch-project#5618).
Signed-off-by: Kai Huang <ahkcs@amazon.com>
@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
22f7212 to
3f0f955CompareAugust 31, 2026 22:40
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 3f0f955
@dai-chen
dai-chen
commented
Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Related to #5610
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@RyanL1997RyanL1997RyanL1997 left review comments
@ps48ps48Awaiting requested review from ps48ps48 is a code owner
@joshuali925joshuali925Awaiting requested review from joshuali925joshuali925 is a code owner
@dai-chendai-chenAwaiting requested review from dai-chendai-chen is a code owner
@mengweiericmengweiericAwaiting requested review from mengweiericmengweieric is a code owner
@vamsimanoharvamsimanoharAwaiting requested review from vamsimanoharvamsimanohar is a code owner
@SwiddisSwiddisAwaiting requested review from SwiddisSwiddis is a code owner
@penghuopenghuoAwaiting requested review from penghuopenghuo is a code owner
@anirudhaanirudhaAwaiting requested review from anirudhaanirudha is a code owner
@acarbonettoacarbonettoAwaiting requested review from acarbonettoacarbonetto is a code owner
@ykmr1224ykmr1224Awaiting requested review from ykmr1224ykmr1224 is a code owner
@LantaoJinLantaoJinAwaiting requested review from LantaoJinLantaoJin is a code owner
@noChargernoChargerAwaiting requested review from noChargernoCharger is a code owner
@qianheng-awsqianheng-awsAwaiting requested review from qianheng-awsqianheng-aws is a code owner
@yuancuyuancuAwaiting requested review from yuancuyuancu is a code owner
@songkant-awssongkant-awsAwaiting requested review from songkant-awssongkant-aws is a code owner

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels

bugFix

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[BUG] dedup on a dotted object-hierarchy field throws "String cannot be cast to java.util.Map" (Calcite path)


3 participants

@ahkcs@dai-chen@RyanL1997

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) by ahkcs · Pull Request #5732 · opensearch-project/sql · GitHub
Skip to content
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) - #5732
Open
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Open
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685)#5732
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Conversation
@ahkcs
@ahkcsahkcs
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
On the Calcite path, PPL dedup pushes down as a composite aggregation whose surviving row is fetched via top_hits and decoded from the nested _source. When a query spans a wildcard/alias over indices that disagree on whether a path is an object or a scalar, the cross-index mapping merge types the path as an object (STRUCT) while some documents store a scalar there. Decoding that scalar as a struct threw and failed the whole query:
java.sql.SQLException: ... class java.lang.String cannot be cast to class java.util.Map

This guards the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a scalar-under-object value to null, completing the graceful handling #5618 added for the geo_point and scalar branches (it did not cover the object/STRUCT branch). The catch is scoped to ClassCastException — this branch is also the whole-document parse entry point and each nested object level is wrapped independently, so a broader catch would swallow legitimate errors raised deeper in the recursion.
Issues Resolved
Closes#5685
Behavior (before → after)
ScenarioBeforeAfter
dedup over wildcard, object-vs-scalar conflict on an object path500 String cannot be cast to Mapsucceeds; conflicting field → null
dedup over wildcard, scalar-vs-object conflict (mirror)already handled by #5618unchanged (regression-guarded)
Unsupported-type parse, deeply nestedIllegalStateException (correct)unchanged (not masked)
Testing
  • Unit: OpenSearchExprValueFactoryTest.constructStructWithScalarValueReturnsNull
  • YAML REST: rest-api-spec/test/issues/5685.yml — object-vs-scalar conflict and its mirror (scalar-vs-object).
  • Full OpenSearchExprValueFactoryTest and the :opensearch unit suite pass; integ-test compiles.
Check List
  •  New functionality includes testing.
  •  Commits are signed per the DCO using --signoff.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Reviewer Guide 🔍
(Review updated until commit 3f0f955)
Here are some key observations to aid the review process:
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Code Suggestions ✨
Latest suggestions up to 3f0f955
Explore these optional code suggestions:
CategorySuggestion Impact
General
Add logging for caught exceptions
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production environments while 
maintaining the graceful degradation behavior.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value found under object-typed field '{}', returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException could aid debugging, the suggestion doesn't address a critical issue. The PR already handles the exception gracefully, and adding logging is a minor enhancement for observability rather than a functional improvement.
Low

Previous suggestions
Suggestions up to commit 22f7212
CategorySuggestion Impact
General
Log caught ClassCastException for debugging
Catching ClassCastException silently may hide legitimate bugs in parseStruct or its 
callees. Consider logging the exception with context (field name, content type) to 
aid debugging while still returning null for the known scalar-under-object scenario.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value encountered under object-typed field '{}': {}", field, e.getMessage());
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: Adding logging for the caught ClassCastException would help with debugging and understanding when this edge case occurs. However, the comment in the code already explains this is a specific fix for issue #5685 (scalar under object-typed field), and the catch is intentionally scoped to avoid hiding other errors. The suggestion is valid but offers moderate improvement since the code already has explanatory comments.
Low

Suggestions up to commit 7d340e8
CategorySuggestion Impact
General
Add logging for mapping conflicts
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production without failing 
queries, providing visibility into when scalar-to-struct coercion occurs.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [234-238]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Field '{}' mapped as object but contains scalar value, returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException would provide useful diagnostic information for mapping conflicts, this is a minor enhancement rather than a critical fix. The suggestion is valid and would improve observability, but the current implementation already handles the error gracefully by returning null.
Low

@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
7d340e8 to
22f7212CompareAugust 31, 2026 22:06
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 22f7212
RyanL1997
RyanL1997
reviewed
Aug 31, 2026
View reviewed changes
Comment thread...earch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java
Outdated

 || type == STRUCT) {
 return parseStruct(content, field, supportArrays);
 // A scalar under an object-typed field (wildcard over indices with conflicting mappings)
 // would throw here; return null instead (#5685). CCE-scoped: this is also the whole-document
@RyanL1997RyanL1997Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets just remove the inline PR link - instead if it is necessary lets to the yaml test with issue id.
@ahkcsahkcsAug 31, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use yaml test as IT
@ahkcs
Fix dedup 500 on cross-index object/scalar mapping conflict (opensear…
3f0f955
…ch-project#5685)
On the Calcite path, `dedup` pushes down as a composite aggregation whose
surviving row is fetched via top_hits and decoded from the nested _source.
When a wildcard spans indices that disagree on whether a path is an object
or a scalar, the merged mapping types the path as an object (STRUCT) while
some documents store a scalar there. Decoding that scalar as a struct threw
`java.sql.SQLException: ... class java.lang.String cannot be cast to
class java.util.Map`, failing the whole query (100% on the reporting
customer's `source=logs-pr172502-*`).
Guard the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a
scalar-under-object value to null, completing the graceful handling opensearch-project#5618
added for the geo_point and scalar branches. The catch is scoped to
ClassCastException because this branch is also the whole-document parse
entry point and each nested object level is wrapped independently, so a
broader catch would swallow legitimate errors raised deeper in the recursion.
Adds a factory unit regression and a yamlRestTest (issues/5685.yml) covering
the object-vs-scalar conflict and its mirror (scalar-vs-object, guarding opensearch-project#5618).
Signed-off-by: Kai Huang <ahkcs@amazon.com>
@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
22f7212 to
3f0f955CompareAugust 31, 2026 22:40
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 3f0f955
@dai-chen
dai-chen
commented
Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Related to #5610
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@RyanL1997RyanL1997RyanL1997 left review comments
@ps48ps48Awaiting requested review from ps48ps48 is a code owner
@joshuali925joshuali925Awaiting requested review from joshuali925joshuali925 is a code owner
@dai-chendai-chenAwaiting requested review from dai-chendai-chen is a code owner
@mengweiericmengweiericAwaiting requested review from mengweiericmengweieric is a code owner
@vamsimanoharvamsimanoharAwaiting requested review from vamsimanoharvamsimanohar is a code owner
@SwiddisSwiddisAwaiting requested review from SwiddisSwiddis is a code owner
@penghuopenghuoAwaiting requested review from penghuopenghuo is a code owner
@anirudhaanirudhaAwaiting requested review from anirudhaanirudha is a code owner
@acarbonettoacarbonettoAwaiting requested review from acarbonettoacarbonetto is a code owner
@ykmr1224ykmr1224Awaiting requested review from ykmr1224ykmr1224 is a code owner
@LantaoJinLantaoJinAwaiting requested review from LantaoJinLantaoJin is a code owner
@noChargernoChargerAwaiting requested review from noChargernoCharger is a code owner
@qianheng-awsqianheng-awsAwaiting requested review from qianheng-awsqianheng-aws is a code owner
@yuancuyuancuAwaiting requested review from yuancuyuancu is a code owner
@songkant-awssongkant-awsAwaiting requested review from songkant-awssongkant-aws is a code owner

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels

bugFix

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[BUG] dedup on a dotted object-hierarchy field throws "String cannot be cast to java.util.Map" (Calcite path)


3 participants

@ahkcs@dai-chen@RyanL1997

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) by ahkcs · Pull Request #5732 · opensearch-project/sql · GitHub
Skip to content
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) - #5732
Open
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Open
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685)#5732
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Conversation
@ahkcs
@ahkcsahkcs
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
On the Calcite path, PPL dedup pushes down as a composite aggregation whose surviving row is fetched via top_hits and decoded from the nested _source. When a query spans a wildcard/alias over indices that disagree on whether a path is an object or a scalar, the cross-index mapping merge types the path as an object (STRUCT) while some documents store a scalar there. Decoding that scalar as a struct threw and failed the whole query:
java.sql.SQLException: ... class java.lang.String cannot be cast to class java.util.Map

This guards the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a scalar-under-object value to null, completing the graceful handling #5618 added for the geo_point and scalar branches (it did not cover the object/STRUCT branch). The catch is scoped to ClassCastException — this branch is also the whole-document parse entry point and each nested object level is wrapped independently, so a broader catch would swallow legitimate errors raised deeper in the recursion.
Issues Resolved
Closes#5685
Behavior (before → after)
ScenarioBeforeAfter
dedup over wildcard, object-vs-scalar conflict on an object path500 String cannot be cast to Mapsucceeds; conflicting field → null
dedup over wildcard, scalar-vs-object conflict (mirror)already handled by #5618unchanged (regression-guarded)
Unsupported-type parse, deeply nestedIllegalStateException (correct)unchanged (not masked)
Testing
  • Unit: OpenSearchExprValueFactoryTest.constructStructWithScalarValueReturnsNull
  • YAML REST: rest-api-spec/test/issues/5685.yml — object-vs-scalar conflict and its mirror (scalar-vs-object).
  • Full OpenSearchExprValueFactoryTest and the :opensearch unit suite pass; integ-test compiles.
Check List
  •  New functionality includes testing.
  •  Commits are signed per the DCO using --signoff.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Reviewer Guide 🔍
(Review updated until commit 3f0f955)
Here are some key observations to aid the review process:
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Code Suggestions ✨
Latest suggestions up to 3f0f955
Explore these optional code suggestions:
CategorySuggestion Impact
General
Add logging for caught exceptions
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production environments while 
maintaining the graceful degradation behavior.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value found under object-typed field '{}', returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException could aid debugging, the suggestion doesn't address a critical issue. The PR already handles the exception gracefully, and adding logging is a minor enhancement for observability rather than a functional improvement.
Low

Previous suggestions
Suggestions up to commit 22f7212
CategorySuggestion Impact
General
Log caught ClassCastException for debugging
Catching ClassCastException silently may hide legitimate bugs in parseStruct or its 
callees. Consider logging the exception with context (field name, content type) to 
aid debugging while still returning null for the known scalar-under-object scenario.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value encountered under object-typed field '{}': {}", field, e.getMessage());
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: Adding logging for the caught ClassCastException would help with debugging and understanding when this edge case occurs. However, the comment in the code already explains this is a specific fix for issue #5685 (scalar under object-typed field), and the catch is intentionally scoped to avoid hiding other errors. The suggestion is valid but offers moderate improvement since the code already has explanatory comments.
Low

Suggestions up to commit 7d340e8
CategorySuggestion Impact
General
Add logging for mapping conflicts
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production without failing 
queries, providing visibility into when scalar-to-struct coercion occurs.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [234-238]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Field '{}' mapped as object but contains scalar value, returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException would provide useful diagnostic information for mapping conflicts, this is a minor enhancement rather than a critical fix. The suggestion is valid and would improve observability, but the current implementation already handles the error gracefully by returning null.
Low

@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
7d340e8 to
22f7212CompareAugust 31, 2026 22:06
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 22f7212
RyanL1997
RyanL1997
reviewed
Aug 31, 2026
View reviewed changes
Comment thread...earch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java
Outdated

 || type == STRUCT) {
 return parseStruct(content, field, supportArrays);
 // A scalar under an object-typed field (wildcard over indices with conflicting mappings)
 // would throw here; return null instead (#5685). CCE-scoped: this is also the whole-document
@RyanL1997RyanL1997Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets just remove the inline PR link - instead if it is necessary lets to the yaml test with issue id.
@ahkcsahkcsAug 31, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use yaml test as IT
@ahkcs
Fix dedup 500 on cross-index object/scalar mapping conflict (opensear…
3f0f955
…ch-project#5685)
On the Calcite path, `dedup` pushes down as a composite aggregation whose
surviving row is fetched via top_hits and decoded from the nested _source.
When a wildcard spans indices that disagree on whether a path is an object
or a scalar, the merged mapping types the path as an object (STRUCT) while
some documents store a scalar there. Decoding that scalar as a struct threw
`java.sql.SQLException: ... class java.lang.String cannot be cast to
class java.util.Map`, failing the whole query (100% on the reporting
customer's `source=logs-pr172502-*`).
Guard the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a
scalar-under-object value to null, completing the graceful handling opensearch-project#5618
added for the geo_point and scalar branches. The catch is scoped to
ClassCastException because this branch is also the whole-document parse
entry point and each nested object level is wrapped independently, so a
broader catch would swallow legitimate errors raised deeper in the recursion.
Adds a factory unit regression and a yamlRestTest (issues/5685.yml) covering
the object-vs-scalar conflict and its mirror (scalar-vs-object, guarding opensearch-project#5618).
Signed-off-by: Kai Huang <ahkcs@amazon.com>
@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
22f7212 to
3f0f955CompareAugust 31, 2026 22:40
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 3f0f955
@dai-chen
dai-chen
commented
Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Related to #5610
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@RyanL1997RyanL1997RyanL1997 left review comments
@ps48ps48Awaiting requested review from ps48ps48 is a code owner
@joshuali925joshuali925Awaiting requested review from joshuali925joshuali925 is a code owner
@dai-chendai-chenAwaiting requested review from dai-chendai-chen is a code owner
@mengweiericmengweiericAwaiting requested review from mengweiericmengweieric is a code owner
@vamsimanoharvamsimanoharAwaiting requested review from vamsimanoharvamsimanohar is a code owner
@SwiddisSwiddisAwaiting requested review from SwiddisSwiddis is a code owner
@penghuopenghuoAwaiting requested review from penghuopenghuo is a code owner
@anirudhaanirudhaAwaiting requested review from anirudhaanirudha is a code owner
@acarbonettoacarbonettoAwaiting requested review from acarbonettoacarbonetto is a code owner
@ykmr1224ykmr1224Awaiting requested review from ykmr1224ykmr1224 is a code owner
@LantaoJinLantaoJinAwaiting requested review from LantaoJinLantaoJin is a code owner
@noChargernoChargerAwaiting requested review from noChargernoCharger is a code owner
@qianheng-awsqianheng-awsAwaiting requested review from qianheng-awsqianheng-aws is a code owner
@yuancuyuancuAwaiting requested review from yuancuyuancu is a code owner
@songkant-awssongkant-awsAwaiting requested review from songkant-awssongkant-aws is a code owner

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels

bugFix

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[BUG] dedup on a dotted object-hierarchy field throws "String cannot be cast to java.util.Map" (Calcite path)


3 participants

@ahkcs@dai-chen@RyanL1997

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) by ahkcs · Pull Request #5732 · opensearch-project/sql · GitHub
Skip to content
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) - #5732
Open
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Open
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685)#5732
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Conversation
@ahkcs
@ahkcsahkcs
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
On the Calcite path, PPL dedup pushes down as a composite aggregation whose surviving row is fetched via top_hits and decoded from the nested _source. When a query spans a wildcard/alias over indices that disagree on whether a path is an object or a scalar, the cross-index mapping merge types the path as an object (STRUCT) while some documents store a scalar there. Decoding that scalar as a struct threw and failed the whole query:
java.sql.SQLException: ... class java.lang.String cannot be cast to class java.util.Map

This guards the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a scalar-under-object value to null, completing the graceful handling #5618 added for the geo_point and scalar branches (it did not cover the object/STRUCT branch). The catch is scoped to ClassCastException — this branch is also the whole-document parse entry point and each nested object level is wrapped independently, so a broader catch would swallow legitimate errors raised deeper in the recursion.
Issues Resolved
Closes#5685
Behavior (before → after)
ScenarioBeforeAfter
dedup over wildcard, object-vs-scalar conflict on an object path500 String cannot be cast to Mapsucceeds; conflicting field → null
dedup over wildcard, scalar-vs-object conflict (mirror)already handled by #5618unchanged (regression-guarded)
Unsupported-type parse, deeply nestedIllegalStateException (correct)unchanged (not masked)
Testing
  • Unit: OpenSearchExprValueFactoryTest.constructStructWithScalarValueReturnsNull
  • YAML REST: rest-api-spec/test/issues/5685.yml — object-vs-scalar conflict and its mirror (scalar-vs-object).
  • Full OpenSearchExprValueFactoryTest and the :opensearch unit suite pass; integ-test compiles.
Check List
  •  New functionality includes testing.
  •  Commits are signed per the DCO using --signoff.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Reviewer Guide 🔍
(Review updated until commit 3f0f955)
Here are some key observations to aid the review process:
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Code Suggestions ✨
Latest suggestions up to 3f0f955
Explore these optional code suggestions:
CategorySuggestion Impact
General
Add logging for caught exceptions
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production environments while 
maintaining the graceful degradation behavior.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value found under object-typed field '{}', returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException could aid debugging, the suggestion doesn't address a critical issue. The PR already handles the exception gracefully, and adding logging is a minor enhancement for observability rather than a functional improvement.
Low

Previous suggestions
Suggestions up to commit 22f7212
CategorySuggestion Impact
General
Log caught ClassCastException for debugging
Catching ClassCastException silently may hide legitimate bugs in parseStruct or its 
callees. Consider logging the exception with context (field name, content type) to 
aid debugging while still returning null for the known scalar-under-object scenario.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value encountered under object-typed field '{}': {}", field, e.getMessage());
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: Adding logging for the caught ClassCastException would help with debugging and understanding when this edge case occurs. However, the comment in the code already explains this is a specific fix for issue #5685 (scalar under object-typed field), and the catch is intentionally scoped to avoid hiding other errors. The suggestion is valid but offers moderate improvement since the code already has explanatory comments.
Low

Suggestions up to commit 7d340e8
CategorySuggestion Impact
General
Add logging for mapping conflicts
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production without failing 
queries, providing visibility into when scalar-to-struct coercion occurs.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [234-238]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Field '{}' mapped as object but contains scalar value, returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException would provide useful diagnostic information for mapping conflicts, this is a minor enhancement rather than a critical fix. The suggestion is valid and would improve observability, but the current implementation already handles the error gracefully by returning null.
Low

@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
7d340e8 to
22f7212CompareAugust 31, 2026 22:06
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 22f7212
RyanL1997
RyanL1997
reviewed
Aug 31, 2026
View reviewed changes
Comment thread...earch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java
Outdated

 || type == STRUCT) {
 return parseStruct(content, field, supportArrays);
 // A scalar under an object-typed field (wildcard over indices with conflicting mappings)
 // would throw here; return null instead (#5685). CCE-scoped: this is also the whole-document
@RyanL1997RyanL1997Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets just remove the inline PR link - instead if it is necessary lets to the yaml test with issue id.
@ahkcsahkcsAug 31, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use yaml test as IT
@ahkcs
Fix dedup 500 on cross-index object/scalar mapping conflict (opensear…
3f0f955
…ch-project#5685)
On the Calcite path, `dedup` pushes down as a composite aggregation whose
surviving row is fetched via top_hits and decoded from the nested _source.
When a wildcard spans indices that disagree on whether a path is an object
or a scalar, the merged mapping types the path as an object (STRUCT) while
some documents store a scalar there. Decoding that scalar as a struct threw
`java.sql.SQLException: ... class java.lang.String cannot be cast to
class java.util.Map`, failing the whole query (100% on the reporting
customer's `source=logs-pr172502-*`).
Guard the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a
scalar-under-object value to null, completing the graceful handling opensearch-project#5618
added for the geo_point and scalar branches. The catch is scoped to
ClassCastException because this branch is also the whole-document parse
entry point and each nested object level is wrapped independently, so a
broader catch would swallow legitimate errors raised deeper in the recursion.
Adds a factory unit regression and a yamlRestTest (issues/5685.yml) covering
the object-vs-scalar conflict and its mirror (scalar-vs-object, guarding opensearch-project#5618).
Signed-off-by: Kai Huang <ahkcs@amazon.com>
@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
22f7212 to
3f0f955CompareAugust 31, 2026 22:40
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 3f0f955
@dai-chen
dai-chen
commented
Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Related to #5610
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@RyanL1997RyanL1997RyanL1997 left review comments
@ps48ps48Awaiting requested review from ps48ps48 is a code owner
@joshuali925joshuali925Awaiting requested review from joshuali925joshuali925 is a code owner
@dai-chendai-chenAwaiting requested review from dai-chendai-chen is a code owner
@mengweiericmengweiericAwaiting requested review from mengweiericmengweieric is a code owner
@vamsimanoharvamsimanoharAwaiting requested review from vamsimanoharvamsimanohar is a code owner
@SwiddisSwiddisAwaiting requested review from SwiddisSwiddis is a code owner
@penghuopenghuoAwaiting requested review from penghuopenghuo is a code owner
@anirudhaanirudhaAwaiting requested review from anirudhaanirudha is a code owner
@acarbonettoacarbonettoAwaiting requested review from acarbonettoacarbonetto is a code owner
@ykmr1224ykmr1224Awaiting requested review from ykmr1224ykmr1224 is a code owner
@LantaoJinLantaoJinAwaiting requested review from LantaoJinLantaoJin is a code owner
@noChargernoChargerAwaiting requested review from noChargernoCharger is a code owner
@qianheng-awsqianheng-awsAwaiting requested review from qianheng-awsqianheng-aws is a code owner
@yuancuyuancuAwaiting requested review from yuancuyuancu is a code owner
@songkant-awssongkant-awsAwaiting requested review from songkant-awssongkant-aws is a code owner

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels

bugFix

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[BUG] dedup on a dotted object-hierarchy field throws "String cannot be cast to java.util.Map" (Calcite path)


3 participants

@ahkcs@dai-chen@RyanL1997

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) by ahkcs · Pull Request #5732 · opensearch-project/sql · GitHub
Skip to content
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685) - #5732
Open
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Open
fix: gracefully handle scalar values under object-typed fields when decoding results (#5685)#5732
ahkcs wants to merge 1 commit into
opensearch-project:mainfrom 
ahkcs:fix/dedup-object-scalar-mapping-5685
Conversation
@ahkcs
@ahkcsahkcs
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Collaborator
Description
On the Calcite path, PPL dedup pushes down as a composite aggregation whose surviving row is fetched via top_hits and decoded from the nested _source. When a query spans a wildcard/alias over indices that disagree on whether a path is an object or a scalar, the cross-index mapping merge types the path as an object (STRUCT) while some documents store a scalar there. Decoding that scalar as a struct threw and failed the whole query:
java.sql.SQLException: ... class java.lang.String cannot be cast to class java.util.Map

This guards the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a scalar-under-object value to null, completing the graceful handling #5618 added for the geo_point and scalar branches (it did not cover the object/STRUCT branch). The catch is scoped to ClassCastException — this branch is also the whole-document parse entry point and each nested object level is wrapped independently, so a broader catch would swallow legitimate errors raised deeper in the recursion.
Issues Resolved
Closes#5685
Behavior (before → after)
ScenarioBeforeAfter
dedup over wildcard, object-vs-scalar conflict on an object path500 String cannot be cast to Mapsucceeds; conflicting field → null
dedup over wildcard, scalar-vs-object conflict (mirror)already handled by #5618unchanged (regression-guarded)
Unsupported-type parse, deeply nestedIllegalStateException (correct)unchanged (not masked)
Testing
  • Unit: OpenSearchExprValueFactoryTest.constructStructWithScalarValueReturnsNull
  • YAML REST: rest-api-spec/test/issues/5685.yml — object-vs-scalar conflict and its mirror (scalar-vs-object).
  • Full OpenSearchExprValueFactoryTest and the :opensearch unit suite pass; integ-test compiles.
Check List
  •  New functionality includes testing.
  •  Commits are signed per the DCO using --signoff.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Reviewer Guide 🔍
(Review updated until commit 3f0f955)
Here are some key observations to aid the review process:
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected
@github-actions
github-actionsBot
commented
Aug 31, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
PR Code Suggestions ✨
Latest suggestions up to 3f0f955
Explore these optional code suggestions:
CategorySuggestion Impact
General
Add logging for caught exceptions
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production environments while 
maintaining the graceful degradation behavior.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value found under object-typed field '{}', returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException could aid debugging, the suggestion doesn't address a critical issue. The PR already handles the exception gracefully, and adding logging is a minor enhancement for observability rather than a functional improvement.
Low

Previous suggestions
Suggestions up to commit 22f7212
CategorySuggestion Impact
General
Log caught ClassCastException for debugging
Catching ClassCastException silently may hide legitimate bugs in parseStruct or its 
callees. Consider logging the exception with context (field name, content type) to 
aid debugging while still returning null for the known scalar-under-object scenario.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [228-232]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Scalar value encountered under object-typed field '{}': {}", field, e.getMessage());
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: Adding logging for the caught ClassCastException would help with debugging and understanding when this edge case occurs. However, the comment in the code already explains this is a specific fix for issue #5685 (scalar under object-typed field), and the catch is intentionally scoped to avoid hiding other errors. The suggestion is valid but offers moderate improvement since the code already has explanatory comments.
Low

Suggestions up to commit 7d340e8
CategorySuggestion Impact
General
Add logging for mapping conflicts
Consider logging the ClassCastException at debug or trace level before returning 
null. This would help diagnose mapping conflicts in production without failing 
queries, providing visibility into when scalar-to-struct coercion occurs.
opensearch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java [234-238]
 try {
return parseStruct(content, field, supportArrays);
} catch (ClassCastException e) {
+ log.debug("Field '{}' mapped as object but contains scalar value, returning null", field, e);
return ExprNullValue.of();
}
Suggestion importance[1-10]: 5
__
Why: While logging the ClassCastException would provide useful diagnostic information for mapping conflicts, this is a minor enhancement rather than a critical fix. The suggestion is valid and would improve observability, but the current implementation already handles the error gracefully by returning null.
Low

@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
7d340e8 to
22f7212CompareAugust 31, 2026 22:06
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 22f7212
RyanL1997
RyanL1997
reviewed
Aug 31, 2026
View reviewed changes
Comment thread...earch/src/main/java/org/opensearch/sql/opensearch/data/value/OpenSearchExprValueFactory.java
Outdated

 || type == STRUCT) {
 return parseStruct(content, field, supportArrays);
 // A scalar under an object-typed field (wildcard over indices with conflicting mappings)
 // would throw here; return null instead (#5685). CCE-scoped: this is also the whole-document
@RyanL1997RyanL1997Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets just remove the inline PR link - instead if it is necessary lets to the yaml test with issue id.
@ahkcsahkcsAug 31, 2026

Copy link


Copy Markdown

CollaboratorAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to use yaml test as IT
@ahkcs
Fix dedup 500 on cross-index object/scalar mapping conflict (opensear…
3f0f955
…ch-project#5685)
On the Calcite path, `dedup` pushes down as a composite aggregation whose
surviving row is fetched via top_hits and decoded from the nested _source.
When a wildcard spans indices that disagree on whether a path is an object
or a scalar, the merged mapping types the path as an object (STRUCT) while
some documents store a scalar there. Decoding that scalar as a struct threw
`java.sql.SQLException: ... class java.lang.String cannot be cast to
class java.util.Map`, failing the whole query (100% on the reporting
customer's `source=logs-pr172502-*`).
Guard the STRUCT branch of OpenSearchExprValueFactory.parse to degrade a
scalar-under-object value to null, completing the graceful handling opensearch-project#5618
added for the geo_point and scalar branches. The catch is scoped to
ClassCastException because this branch is also the whole-document parse
entry point and each nested object level is wrapped independently, so a
broader catch would swallow legitimate errors raised deeper in the recursion.
Adds a factory unit regression and a yamlRestTest (issues/5685.yml) covering
the object-vs-scalar conflict and its mirror (scalar-vs-object, guarding opensearch-project#5618).
Signed-off-by: Kai Huang <ahkcs@amazon.com>
@ahkcs
ahkcsforce-pushed
the
fix/dedup-object-scalar-mapping-5685
branch
from
22f7212 to
3f0f955CompareAugust 31, 2026 22:40
@github-actions
github-actionsBot
commented
Aug 31, 2026

Copy link


Copy Markdown

Contributor
Persistent review updated to latest commit 3f0f955
@dai-chen
dai-chen
commented
Aug 31, 2026

Copy link


Copy Markdown

Collaborator
Related to #5610
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@RyanL1997RyanL1997RyanL1997 left review comments
@ps48ps48Awaiting requested review from ps48ps48 is a code owner
@joshuali925joshuali925Awaiting requested review from joshuali925joshuali925 is a code owner
@dai-chendai-chenAwaiting requested review from dai-chendai-chen is a code owner
@mengweiericmengweiericAwaiting requested review from mengweiericmengweieric is a code owner
@vamsimanoharvamsimanoharAwaiting requested review from vamsimanoharvamsimanohar is a code owner
@SwiddisSwiddisAwaiting requested review from SwiddisSwiddis is a code owner
@penghuopenghuoAwaiting requested review from penghuopenghuo is a code owner
@anirudhaanirudhaAwaiting requested review from anirudhaanirudha is a code owner
@acarbonettoacarbonettoAwaiting requested review from acarbonettoacarbonetto is a code owner
@ykmr1224ykmr1224Awaiting requested review from ykmr1224ykmr1224 is a code owner
@LantaoJinLantaoJinAwaiting requested review from LantaoJinLantaoJin is a code owner
@noChargernoChargerAwaiting requested review from noChargernoCharger is a code owner
@qianheng-awsqianheng-awsAwaiting requested review from qianheng-awsqianheng-aws is a code owner
@yuancuyuancuAwaiting requested review from yuancuyuancu is a code owner
@songkant-awssongkant-awsAwaiting requested review from songkant-awssongkant-aws is a code owner

At least 1 approving review is required to merge this pull request.


Assignees


No one assigned


Labels

bugFix

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[BUG] dedup on a dotted object-hierarchy field throws "String cannot be cast to java.util.Map" (Calcite path)


3 participants

@ahkcs@dai-chen@RyanL1997