[Fix](JsonReader) Fix the issue where the null bitmap of the JSON reader was not initialized when the JSON path is specified as '$.’ by eldenmoon · Pull Request #52211 · apache/doris

eldenmoon · 2025-06-24T08:40:27Z

The null bitmap of the related json path should be inited

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-06-24T08:40:56Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

…der was not initialized when the JSON path is specified as '$.’.

eldenmoon · 2025-06-24T08:43:39Z

run buildall

doris-robot · 2025-06-24T10:01:03Z

TPC-H: Total hot run time: 34038 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7fe7c7ab1ce1520aab69e6c3cf6da0ec72e2cb4e, data reload: false
------ Round 1 ----------------------------------
q1	17597	5302	5008	5008
q2	1941	307	192	192
q3	10292	1301	728	728
q4	10226	1020	550	550
q5	7530	2270	2473	2270
q6	183	161	134	134
q7	899	759	617	617
q8	9342	1340	1191	1191
q9	6812	5079	5028	5028
q10	6895	2394	1960	1960
q11	488	297	285	285
q12	347	351	220	220
q13	17770	3686	3153	3153
q14	235	222	217	217
q15	552	475	476	475
q16	431	425	379	379
q17	597	880	370	370
q18	8000	7346	7193	7193
q19	1478	959	543	543
q20	349	336	230	230
q21	3687	3256	2341	2341
q22	1032	1029	954	954
Total cold run time: 106683 ms
Total hot run time: 34038 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5118	5097	5135	5097
q2	247	322	222	222
q3	2214	2845	2351	2351
q4	1342	1813	1331	1331
q5	4265	4424	4488	4424
q6	220	173	128	128
q7	1981	1900	1791	1791
q8	2582	2581	2569	2569
q9	7170	7179	7110	7110
q10	3046	3257	2818	2818
q11	581	507	490	490
q12	661	768	631	631
q13	3542	3938	3329	3329
q14	280	298	282	282
q15	533	495	491	491
q16	444	513	450	450
q17	1187	1554	1332	1332
q18	7310	7231	7126	7126
q19	760	761	805	761
q20	1913	1998	1802	1802
q21	4834	4393	4362	4362
q22	1062	1056	988	988
Total cold run time: 51292 ms
Total hot run time: 49885 ms

doris-robot · 2025-06-24T10:12:38Z

TPC-DS: Total hot run time: 185787 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7fe7c7ab1ce1520aab69e6c3cf6da0ec72e2cb4e, data reload: false
query1	1020	391	412	391
query2	6530	1896	1865	1865
query3	6739	229	221	221
query4	26223	23492	23066	23066
query5	4325	637	467	467
query6	291	209	191	191
query7	4621	510	287	287
query8	264	223	204	204
query9	8613	2581	2594	2581
query10	472	333	259	259
query11	15379	15132	14886	14886
query12	160	107	105	105
query13	1654	541	393	393
query14	9507	6172	6345	6172
query15	204	195	171	171
query16	7453	621	493	493
query17	1236	734	638	638
query18	2012	407	290	290
query19	189	192	159	159
query20	120	115	112	112
query21	212	124	103	103
query22	4176	4276	4086	4086
query23	34155	33155	33071	33071
query24	8464	2369	2431	2369
query25	538	458	381	381
query26	1226	265	149	149
query27	2754	519	342	342
query28	4282	2125	2104	2104
query29	754	540	434	434
query30	296	226	218	218
query31	942	845	775	775
query32	73	66	61	61
query33	564	369	316	316
query34	820	863	524	524
query35	765	834	747	747
query36	947	974	883	883
query37	113	98	82	82
query38	4202	4141	4023	4023
query39	1494	1496	1413	1413
query40	219	116	105	105
query41	63	58	66	58
query42	125	110	110	110
query43	515	530	497	497
query44	1316	824	832	824
query45	178	176	166	166
query46	844	1024	626	626
query47	1767	1819	1733	1733
query48	400	436	329	329
query49	736	491	392	392
query50	638	678	409	409
query51	4132	4149	4167	4149
query52	117	106	102	102
query53	227	251	189	189
query54	582	572	499	499
query55	93	97	83	83
query56	313	321	294	294
query57	1218	1185	1118	1118
query58	276	268	270	268
query59	2676	2752	2652	2652
query60	351	343	331	331
query61	162	147	149	147
query62	814	724	682	682
query63	223	218	185	185
query64	4309	1012	679	679
query65	4261	4195	4183	4183
query66	1106	418	322	322
query67	15688	15589	15323	15323
query68	7853	896	540	540
query69	465	307	271	271
query70	1178	1123	1113	1113
query71	457	330	295	295
query72	5324	4707	4619	4619
query73	679	569	358	358
query74	8887	8861	8893	8861
query75	3864	3196	2671	2671
query76	3710	1196	753	753
query77	788	385	296	296
query78	10047	10168	9432	9432
query79	2337	872	583	583
query80	645	522	452	452
query81	502	262	227	227
query82	187	126	101	101
query83	256	269	234	234
query84	248	103	85	85
query85	759	351	314	314
query86	378	309	297	297
query87	4410	4544	4273	4273
query88	3619	2297	2283	2283
query89	411	319	289	289
query90	1960	215	221	215
query91	141	139	111	111
query92	85	62	61	61
query93	2182	946	589	589
query94	633	411	295	295
query95	373	296	287	287
query96	487	570	284	284
query97	2641	2740	2632	2632
query98	231	214	204	204
query99	1324	1431	1275	1275
Total cold run time: 274199 ms
Total hot run time: 185787 ms

doris-robot · 2025-06-24T10:17:55Z

ClickBench: Total hot run time: 29.62 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7fe7c7ab1ce1520aab69e6c3cf6da0ec72e2cb4e, data reload: false
query1	0.03	0.04	0.02
query2	0.08	0.03	0.04
query3	0.25	0.06	0.07
query4	1.64	0.10	0.10
query5	0.44	0.44	0.41
query6	1.16	0.66	0.68
query7	0.03	0.02	0.02
query8	0.05	0.03	0.04
query9	0.57	0.51	0.52
query10	0.56	0.58	0.57
query11	0.15	0.12	0.11
query12	0.15	0.12	0.11
query13	0.63	0.61	0.61
query14	0.80	0.79	0.82
query15	0.91	0.87	0.89
query16	0.41	0.39	0.39
query17	1.07	1.10	1.09
query18	0.22	0.21	0.22
query19	2.02	1.94	1.92
query20	0.02	0.02	0.01
query21	15.36	0.89	0.53
query22	0.75	1.21	0.68
query23	14.91	1.38	0.66
query24	6.86	1.11	0.73
query25	0.47	0.29	0.06
query26	0.54	0.17	0.15
query27	0.06	0.06	0.06
query28	10.22	0.94	0.45
query29	12.62	4.06	3.41
query30	0.26	0.10	0.06
query31	2.83	0.61	0.39
query32	3.25	0.56	0.48
query33	3.07	3.11	3.16
query34	16.07	5.39	4.72
query35	4.83	4.88	4.81
query36	0.66	0.50	0.47
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.15
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.03	0.03	0.04
Total cold run time: 104.43 s
Total hot run time: 29.62 s

doris-robot · 2025-06-24T11:54:50Z

BE UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	56.93% (15338/26941)
Line Coverage	46.01% (139031/302175)
Region Coverage	45.36% (70432/155260)
Branch Coverage	40.15% (37213/92678)

hello-stephen · 2025-06-24T12:55:40Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	62.30% (16512/26503)
Line Coverage	52.11% (157364/301987)
Region Coverage	49.63% (90435/182226)
Branch Coverage	43.25% (45607/105450)

csun5285

LGTM

github-actions · 2025-06-25T02:22:44Z

PR approved by anyone and no changes requested.

airborne12

LGTM

github-actions · 2025-06-25T02:41:50Z

PR approved by at least one committer and no changes requested.

…der was not initialized when the JSON path is specified as '$.’ (#52211) The null bitmap of the related json path should be inited

…the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 (#52267) Cherry-picked from #52211 Co-authored-by: lihangyu <lihangyu@selectdb.com>

…the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 (#52269) Cherry-picked from #52211 Co-authored-by: lihangyu <lihangyu@selectdb.com>

…the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 (#52268) Cherry-picked from #52211 Co-authored-by: lihangyu <lihangyu@selectdb.com>

…the JSON reader was not initialized when the JSON path is specified as '$.’ apache#52211 (apache#52267) Cherry-picked from apache#52211 Co-authored-by: lihangyu <lihangyu@selectdb.com>

…the JSON reader was not initialized when the JSON path is specified as '$.’ apache#52211 apache#52267 (apache#4255) Cherry-picked from apache#52267

eldenmoon changed the title ~~[Fix](JsonReader) Fix the issue where the null bitmap of the JSON rea…~~[Fix](JsonReader) Fix the issue where the null bitmap of the JSON reader was not initialized when the JSON path is specified as '$.’Jun 24, 2025

[Fix](JsonReader) Fix the issue where the null bitmap of the JSON rea…
7fe7c7a
…der was not initialized when the JSON path is specified as '$.’.

eldenmoon force-pushed the fix-jr-dollar branch from 3f8a63a to 7fe7c7aCompare June 24, 2025 08:42

eldenmoon added dev/2.1.x dev/3.0.x usercase Important user case type label dev/3.1.x labels Jun 24, 2025

csun5285 approved these changes Jun 25, 2025
View reviewed changes

github-actionsBot added the reviewed label Jun 25, 2025

airborne12 approved these changes Jun 25, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 25, 2025

eldenmoon merged commit f9e6543 into apache:masterJun 25, 2025

github-actionsBot mentioned this pull request Jun 25, 2025
branch-3.0: [Fix](JsonReader) Fix the issue where the null bitmap of the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 #52267
Merged

github-actionsBot mentioned this pull request Jun 25, 2025
branch-2.1: [Fix](JsonReader) Fix the issue where the null bitmap of the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 #52268
Merged

github-actionsBot mentioned this pull request Jun 25, 2025
branch-3.1: [Fix](JsonReader) Fix the issue where the null bitmap of the JSON reader was not initialized when the JSON path is specified as '$.’ #52211 #52269
Merged

eldenmoon added the p0_b label Jun 25, 2025

eldenmoon deleted the fix-jr-dollar branch June 25, 2025 08:23

dataroaring added dev/3.0.7-merged and removed dev/3.0.x labels Jun 26, 2025

morrySnow added dev/3.1.0-merged and removed dev/3.1.x labels Jun 26, 2025

yiguolei added dev/2.1.11-merged and removed dev/2.1.x labels Jun 28, 2025

gavinchou mentioned this pull request Aug 15, 2025
Release Note 3.0.7 #54868
Closed

Navigation Menu

Uh oh!

[Fix](JsonReader) Fix the issue where the null bitmap of the JSON reader was not initialized when the JSON path is specified as '$.’ - #52211

[Fix](JsonReader) Fix the issue where the null bitmap of the JSON reader was not initialized when the JSON path is specified as '$.’#52211
eldenmoon merged 1 commit into
apache:masterfrom
eldenmoon:fix-jr-dollar

eldenmoon commented Jun 24, 2025•
edited
Loading

Uh oh!

hello-stephen commented Jun 24, 2025

Uh oh!

eldenmoon commented Jun 24, 2025

Uh oh!

doris-robot commented Jun 24, 2025

Uh oh!

doris-robot commented Jun 24, 2025

Uh oh!

doris-robot commented Jun 24, 2025

Uh oh!

doris-robot commented Jun 24, 2025

Uh oh!

hello-stephen commented Jun 24, 2025

Uh oh!

csun5285 left a comment

Uh oh!

github-actionsBot commented Jun 25, 2025

Uh oh!

airborne12 left a comment

Uh oh!

github-actionsBot commented Jun 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Uh oh!

Conversation

eldenmoon commented Jun 24, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jun 24, 2025

Uh oh!

eldenmoon commented Jun 24, 2025

Uh oh!

doris-robot commented Jun 24, 2025

Uh oh!

doris-robot commented Jun 24, 2025

Uh oh!

doris-robot commented Jun 24, 2025

Uh oh!

doris-robot commented Jun 24, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Jun 24, 2025

BE Regression && UT Coverage Report

Uh oh!

csun5285 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jun 25, 2025

Uh oh!

airborne12 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jun 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

eldenmoon commented Jun 24, 2025•
edited
Loading