Skip to content

[branch-4.0](serde) fix split_by_delimiter missing backslash escape handling (#61995) - #64438

Open
csun5285 wants to merge 1 commit into
apache:branch-4.0from
csun5285:cp-61995-branch-4.0
Open

[branch-4.0](serde) fix split_by_delimiter missing backslash escape handling (#61995)#64438
csun5285 wants to merge 1 commit into
apache:branch-4.0from
csun5285:cp-61995-branch-4.0

Conversation

@csun5285

Copy link
Copy Markdown
Contributor

cherry picked from (#61995)

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@csun5285

Copy link
Copy Markdown
ContributorAuthor

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.38% (25492/35711)
Line Coverage54.25% (270132/497958)
Region Coverage51.80% (223683/431837)
Branch Coverage53.18% (96233/180961)

1 similar comment
@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.38% (25492/35711)
Line Coverage54.25% (270132/497958)
Region Coverage51.80% (223683/431837)
Branch Coverage53.18% (96233/180961)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.38% (25491/35711)
Line Coverage54.25% (270121/497958)
Region Coverage51.79% (223665/431837)
Branch Coverage53.18% (96230/180961)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.38% (25491/35711)
Line Coverage54.24% (270089/497958)
Region Coverage51.77% (223552/431837)
Branch Coverage53.17% (96219/180961)

2 similar comments
@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.38% (25491/35711)
Line Coverage54.24% (270089/497958)
Region Coverage51.77% (223552/431837)
Branch Coverage53.17% (96219/180961)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage71.38% (25491/35711)
Line Coverage54.24% (270089/497958)
Region Coverage51.77% (223552/431837)
Branch Coverage53.17% (96219/180961)

@csun5285csun5285 changed the title [fix](serde) fix split_by_delimiter missing backslash escape handling (#61995)[branch-4.0](serde) fix split_by_delimiter missing backslash escape handling (#61995)Jun 12, 2026
…apache#61995)
Manual backport of apache#61995 to branch-4.0. The serde util still lives under
be/src/vec/data_types/serde/ on this branch (namespace doris::vectorized),
so the core fix and the new UT are applied at the vec/ paths and the test's
added include points to vec/data_types/serde/complex_type_deserialize_util.h.
The core change, the UT body, and the jsonb regression data are unchanged.
### Bug
Stream Load (JSON format) writing to Map<String, String> columns silently
sets the column to NULL when the map value contains both \" (escaped quotes)
and : or ,. Stream Load returns Success with FilteredRows=0 — data loss with
no error indication.
### Fix
Added \ escape skip in split_by_delimiter's character loop, before quote
detection — consistent with the existing escape handling in
deserialize_one_cell_from_json.
(cherry picked from commit 0c9fff3)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@morningman

Copy link
Copy Markdown
Contributor

run buildall

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage57.69% (20619/35742)
Line Coverage40.48% (201739/498360)
Region Coverage37.34% (161439/432317)
Branch Coverage37.97% (68805/181189)

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@csun5285@hello-stephen@morningman