Uh oh!
There was an error while loading. Please reload this page.
branch-4.1: [[fix](variant)](https://github.com/apache/doris/pull/63792) Preserve json object when casting jsonb to variant - #64662
Merged
Conversation
Issue Number: closeapache#24260 Related PR: #xxx Problem Summary: Inserting JSONB-producing expressions, such as json_object or a JSON column, into a VARIANT column could materialize the root JSONB through the generic JSONB-to-string cast path. That path unescaped string contents before variant parsing, so JSON objects containing quoted strings could be parsed incorrectly and subpath extraction failed. This change converts root JSONB directly back to JSON text with JsonbToJson before parsing into ColumnVariant, preserving JSON escapes for object string values. Fix variant insertion from JSON/JSONB expressions for strings containing quotes. - Test: - Build: `./build.sh --be --fe` - Format: `PATH=/mnt/disk1/claude-max/tmp/doris/tmp/clang-format-bin:$PATH build-support/clang-format.sh` with clang-format 16.0.0 - Regression test: `./run-regression-test.sh --run -d variant_p0 -s test_insert_json_object_to_variant -forceGenOut ...` - Regression test: `./run-regression-test.sh --run -d variant_p0 -s test_insert_json_object_to_variant ...` - BE UT: `./run-be-ut.sh --run --filter='SchemaUtilTest.TestParseVariantColumnsEdgeCases:SchemaUtilTest.TestParseJsonbRootVariantMaterializesDocument'` - Manual test: reproduced the affected JSON-to-VARIANT insert SQL before the fix and verified path extraction after the fix - Static check: `CLANG_TIDY_BINARY=/mnt/disk1/claude-max/.vscode-server/extensions/ms-vscode.cpptools-1.32.2-linux-x64/LLVM/bin/clang-tidy build-support/run-clang-tidy.sh --build-dir be/build_Release --files be/src/exec/common/variant_util.cpp` - Behavior changed: Yes. JSON/JSONB values inserted into variant now materialize as JSON objects so subpath extraction works when string values contain quotes. - Does this need documentation: No (cherry picked from commit b6c8054)
hello-stephen
commented
Jun 22, 2026
Contributor
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This cherry-pick fixes VARIANT materialization when inserting JSON/JSONB-derived values whose string contents include quotes/backslashes, by ensuring JSONB roots are converted back to JSON text via a JSONB-aware path before VARIANT parsing/materialization.
Changes:
- Add a JSONB→JSON-text conversion helper and use it when parsing scalar VARIANT roots of type JSONB.
- Add a regression test suite and golden output covering
json_object(...)and JSON column inserts into VARIANT with embedded escaped content. - Strengthen BE unit tests to build real JSONB columns and assert subpath materialization for JSONB-root VARIANT.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| regression-test/suites/variant_p0/test_insert_json_object_to_variant.groovy | Adds regression coverage for inserting JSONB-producing expressions/JSON columns into VARIANT and validating subpath extraction. |
| regression-test/data/variant_p0/test_insert_json_object_to_variant.out | Adds expected output for the new regression queries. |
| be/test/exec/common/schema_util_test.cpp | Updates/extends unit tests to construct real JSONB columns and validate VARIANT materialization/subcolumns. |
| be/src/exec/common/variant_util.cpp | Switches JSONB scalar-root parsing to a JSONB-aware conversion path before VARIANT parsing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+249
to
+258
| auto root_column = root.convert_to_full_column_if_const(); | ||
| const IColumn* jsonb_column = root_column.get(); | ||
| const NullMap* null_map = nullptr; | ||
| if (root_column->is_nullable()) { | ||
| const auto& nullable = assert_cast<const ColumnNullable&>(*root_column); | ||
| jsonb_column = &nullable.get_nested_column(); | ||
| null_map = &nullable.get_null_map_data(); | ||
| } | ||
| const auto& column = assert_cast<const ColumnString&>(*jsonb_column); |
eldenmoon
commented
Jun 22, 2026
MemberAuthor
run buildall |
hello-stephen
commented
Jun 22, 2026
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
hello-stephen
commented
Jun 22, 2026
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
yiguolei
approved these changes
Jun 22, 2026
Uh oh!
There was an error while loading. Please reload this page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cherry-pick #63792