[fix](json-functions)fix json-replace/insert/set/array behavior with … by amorynan · Pull Request #50890 · apache/doris

amorynan · 2025-05-14T07:20:56Z

…complex type (#50308)

here are some functions behavior is not same with mysql as JSON_REPLACE | JSON_INSERT | JSON_SET | JSON_ARRAY
which json and complex types are not supported before this PR Follow mysql :
https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#function_json-set before we will meet result:

mysql> insert into t01 values (1, '{"a":1, "b":2}');
Query OK, 1 row affected (0.25 sec)
{'label':'label_1056456f8d8c46e5_9be38bb344bdd72c', 'status':'VISIBLE', 'txnId':'8019'}
mysql> select b, json_set(b, "$.a", b) from t01;
+---------------+-----------------------+
| b | json_set(b, "$.a", b) |
+---------------+-----------------------+
| {"a":1,"b":2} | {"a":1,"b":2} |
+---------------+-----------------------+

now

mysql> select b, json_set(b, "$.a", b) from t01;
+---------------+---------------------------+
| b | json_set(b, "$.a", b) |
+---------------+---------------------------+
| {"a":1,"b":2} | {"a":{"a":1,"b":2},"b":2} |
+---------------+---------------------------+
1 row in set (0.12 sec)

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

…complex type (apache#50308) here are some functions behavior is not same with mysql as JSON_REPLACE | JSON_INSERT | JSON_SET | JSON_ARRAY which json and complex types are not supported before this PR Follow mysql : https://dev.mysql.com/doc/refman/8.0/en/json-modification-functions.html#function_json-set before we will meet result: ``` mysql> insert into t01 values (1, '{"a":1, "b":2}'); Query OK, 1 row affected (0.25 sec) {'label':'label_1056456f8d8c46e5_9be38bb344bdd72c', 'status':'VISIBLE', 'txnId':'8019'} mysql> select b, json_set(b, "$.a", b) from t01; +---------------+-----------------------+ | b | json_set(b, "$.a", b) | +---------------+-----------------------+ | {"a":1,"b":2} | {"a":1,"b":2} | +---------------+-----------------------+ ``` now ``` mysql> select b, json_set(b, "$.a", b) from t01; +---------------+---------------------------+ | b | json_set(b, "$.a", b) | +---------------+---------------------------+ | {"a":1,"b":2} | {"a":{"a":1,"b":2},"b":2} | +---------------+---------------------------+ 1 row in set (0.12 sec) ```

Thearas · 2025-05-14T07:21:02Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

amorynan · 2025-05-14T07:21:15Z

run buildall

doris-robot · 2025-05-14T07:59:47Z

TPC-H: Total hot run time: 40017 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ce529f114adc4310418f8b5338708008e005cd2a, data reload: false
------ Round 1 ----------------------------------
q1	17576	6895	6557	6557
q2	2069	167	164	164
q3	10621	1055	1156	1055
q4	10560	762	728	728
q5	7733	2874	2831	2831
q6	217	132	133	132
q7	960	607	628	607
q8	9351	1967	1996	1967
q9	6607	6374	6371	6371
q10	7040	2262	2318	2262
q11	491	264	257	257
q12	396	214	207	207
q13	17778	2950	2974	2950
q14	227	200	208	200
q15	496	461	461	461
q16	670	600	599	599
q17	966	539	608	539
q18	7195	6573	6770	6573
q19	1440	1102	1049	1049
q20	480	207	202	202
q21	3946	3308	3299	3299
q22	1096	1023	1007	1007
Total cold run time: 107915 ms
Total hot run time: 40017 ms
----- Round 2, with runtime_filter_mode=off -----
q1	6618	6582	6559	6559
q2	334	235	232	232
q3	2930	2769	2893	2769
q4	2023	1847	1818	1818
q5	5755	5711	5719	5711
q6	210	126	129	126
q7	2237	1797	1836	1797
q8	3389	3531	3487	3487
q9	8891	8834	8919	8834
q10	3571	3510	3502	3502
q11	606	489	487	487
q12	773	583	612	583
q13	8815	3160	3248	3160
q14	294	269	263	263
q15	499	459	473	459
q16	670	658	654	654
q17	1825	1633	1600	1600
q18	8288	7785	7777	7777
q19	1696	1549	1578	1549
q20	2049	1786	1839	1786
q21	5512	5257	5252	5252
q22	1097	1055	1022	1022
Total cold run time: 68082 ms
Total hot run time: 59427 ms

doris-robot · 2025-05-14T08:12:25Z

TPC-DS: Total hot run time: 197806 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ce529f114adc4310418f8b5338708008e005cd2a, data reload: false
query1	1279	894	915	894
query2	6273	2033	2083	2033
query3	10802	4377	4286	4286
query4	61811	30606	23927	23927
query5	5148	471	485	471
query6	435	186	187	186
query7	5438	315	309	309
query8	305	229	232	229
query9	8353	2619	2583	2583
query10	460	272	256	256
query11	17463	15177	15819	15177
query12	166	103	107	103
query13	1420	471	425	425
query14	10882	7115	7618	7115
query15	198	183	178	178
query16	7093	483	489	483
query17	1160	583	574	574
query18	1661	336	308	308
query19	199	164	148	148
query20	124	103	106	103
query21	210	99	99	99
query22	4612	4443	4673	4443
query23	34700	34046	34365	34046
query24	6177	2882	2954	2882
query25	504	389	391	389
query26	645	163	166	163
query27	1888	371	356	356
query28	4220	2474	2422	2422
query29	694	446	451	446
query30	233	159	159	159
query31	1001	831	869	831
query32	64	58	57	57
query33	449	282	283	282
query34	894	498	517	498
query35	848	727	709	709
query36	1079	970	959	959
query37	118	70	69	69
query38	4098	3961	3998	3961
query39	1535	1493	1462	1462
query40	207	100	103	100
query41	50	50	50	50
query42	117	104	106	104
query43	537	518	501	501
query44	1236	825	814	814
query45	198	171	167	167
query46	1154	741	720	720
query47	2025	1895	1935	1895
query48	476	377	419	377
query49	759	422	400	400
query50	833	440	439	439
query51	7403	7233	7308	7233
query52	102	94	96	94
query53	264	188	193	188
query54	596	473	490	473
query55	80	81	80	80
query56	291	271	267	267
query57	1279	1192	1172	1172
query58	239	242	222	222
query59	3304	3170	3138	3138
query60	278	262	272	262
query61	114	113	110	110
query62	754	659	667	659
query63	211	191	185	185
query64	1301	662	635	635
query65	3248	3193	3195	3193
query66	720	293	289	289
query67	15799	15625	15548	15548
query68	4161	586	574	574
query69	439	260	268	260
query70	1151	1079	1083	1079
query71	404	263	254	254
query72	6698	4140	4022	4022
query73	760	350	359	350
query74	10234	9414	9049	9049
query75	3364	2634	2638	2634
query76	1837	1059	1107	1059
query77	492	269	278	269
query78	10679	9661	9533	9533
query79	1836	612	596	596
query80	1407	441	426	426
query81	531	248	242	242
query82	1267	90	86	86
query83	275	143	139	139
query84	284	77	79	77
query85	1023	305	283	283
query86	399	304	303	303
query87	4420	4227	4268	4227
query88	3914	2403	2364	2364
query89	415	297	291	291
query90	2012	183	187	183
query91	187	147	148	147
query92	66	48	51	48
query93	2263	549	551	549
query94	750	277	275	275
query95	355	253	253	253
query96	618	282	275	275
query97	3281	3103	3153	3103
query98	212	200	200	200
query99	1650	1320	1322	1320
Total cold run time: 315598 ms
Total hot run time: 197806 ms

hello-stephen · 2025-05-14T08:13:34Z

BE UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	41.04% (10834/26397)
Line Coverage	31.87% (92473/290145)
Region Coverage	30.94% (47720/154223)
Branch Coverage	27.44% (24447/89106)

doris-robot · 2025-05-14T08:17:53Z

ClickBench: Total hot run time: 32.07 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ce529f114adc4310418f8b5338708008e005cd2a, data reload: false
query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.10
query5	0.54	0.51	0.51
query6	1.14	0.72	0.74
query7	0.02	0.02	0.02
query8	0.03	0.03	0.03
query9	0.57	0.51	0.50
query10	0.55	0.54	0.56
query11	0.14	0.10	0.10
query12	0.13	0.11	0.11
query13	0.60	0.60	0.59
query14	2.82	2.74	2.73
query15	0.89	0.82	0.82
query16	0.39	0.38	0.38
query17	1.03	1.04	1.04
query18	0.24	0.22	0.22
query19	1.80	1.68	2.08
query20	0.02	0.01	0.01
query21	15.36	0.59	0.59
query22	2.62	1.83	2.26
query23	16.95	0.95	0.78
query24	3.25	1.09	1.30
query25	0.30	0.14	0.07
query26	0.57	0.14	0.14
query27	0.04	0.04	0.03
query28	9.94	0.53	0.48
query29	12.58	3.25	3.28
query30	0.25	0.06	0.06
query31	2.86	0.38	0.37
query32	3.28	0.46	0.45
query33	2.97	2.99	3.02
query34	16.88	4.49	4.47
query35	4.54	4.46	4.47
query36	0.68	0.48	0.47
query37	0.09	0.06	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.4 s
Total hot run time: 32.07 s

eldenmoon

LGTM

github-actions · 2025-05-14T09:37:52Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-05-14T09:37:54Z

PR approved by anyone and no changes requested.

dataroaring

LGTM

amorynan requested a review from dataroaring as a code owner May 14, 2025 07:20

eldenmoon approved these changes May 14, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label May 14, 2025

github-actionsBot added the reviewed label May 14, 2025

dataroaring approved these changes May 14, 2025
View reviewed changes

dataroaring merged commit 195cb31 into apache:branch-3.0May 14, 2025

Navigation Menu

Uh oh!

[fix](json-functions)fix json-replace/insert/set/array behavior with … - #50890

[fix](json-functions)fix json-replace/insert/set/array behavior with …#50890
dataroaring merged 1 commit into
apache:branch-3.0from
amorynan:fix-json-funcs

amorynan commented May 14, 2025

Uh oh!

Thearas commented May 14, 2025

Uh oh!

amorynan commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

hello-stephen commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

eldenmoon left a comment

Uh oh!

github-actionsBot commented May 14, 2025

Uh oh!

github-actionsBot commented May 14, 2025

Uh oh!

dataroaring left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

amorynan commented May 14, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented May 14, 2025

Uh oh!

amorynan commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

hello-stephen commented May 14, 2025

BE UT Coverage Report

Uh oh!

doris-robot commented May 14, 2025

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented May 14, 2025

Uh oh!

github-actionsBot commented May 14, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants