Skip to content

[fix](function)fix error result in replace_empty with utf8 chars - #40869

Merged
zhangstar333 merged 1 commit into
apache:masterfrom
Mryange:fix-replace_empty-with-utf8
Sep 19, 2024
Merged

[fix](function)fix error result in replace_empty with utf8 chars#40869
zhangstar333 merged 1 commit into
apache:masterfrom
Mryange:fix-replace_empty-with-utf8

Conversation

@Mryange

@MryangeMryange commented Sep 14, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

mysql [(none)]>select replace_empty('你a好b世c界','','b');
+-------------------------------------------+
| replace_empty('你a好b世c界', '', 'b') |
+-------------------------------------------+
| b�b�b�bab�b�b�bbb�b�b�bcb�b�b�b |
+----------------------------
mysql [(none)]>select replace_empty('你a好b世c界','','b');
+-------------------------------------------+
| replace_empty('你a好b世c界', '', 'b') |
+-------------------------------------------+
| b你bab好bbb世bcb界b |
+-------------------------------------------+

@doris-robot

doris-robot commented Sep 14, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Mryange

Mryange commented Sep 14, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Sep 14, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 36.92% (9475/25664)
Line Coverage: 28.29% (77956/275529)
Region Coverage: 27.69% (40240/145329)
Branch Coverage: 24.29% (20445/84158)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d376e8b9dfadea69b0ab374dbfdd7fea53678537_d376e8b9dfadea69b0ab374dbfdd7fea53678537/report/index.html

@doris-robot

doris-robot commented Sep 14, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 43051 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d376e8b9dfadea69b0ab374dbfdd7fea53678537, data reload: false
------ Round 1 ----------------------------------
q1	17585	7396	7250	7250
q2	2042	190	185	185
q3	10471	1283	1454	1283
q4	10479	1046	1079	1046
q5	7731	3250	3189	3189
q6	243	155	150	150
q7	1052	644	625	625
q8	9458	2074	2077	2074
q9	6897	6355	6332	6332
q10	7038	2511	2518	2511
q11	432	250	253	250
q12	414	228	226	226
q13	17777	3033	3010	3010
q14	292	256	247	247
q15	589	530	543	530
q16	521	425	424	424
q17	1009	957	962	957
q18	7486	6924	6692	6692
q19	1379	1249	1239	1239
q20	607	322	345	322
q21	3926	3518	3540	3518
q22	1097	1016	991	991
Total cold run time: 108525 ms
Total hot run time: 43051 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7172	7191	7210	7191
q2	357	236	230	230
q3	3099	3126	3092	3092
q4	2137	2146	2002	2002
q5	5725	5642	5714	5642
q6	246	153	157	153
q7	2164	1793	1799	1793
q8	3350	3479	3402	3402
q9	8917	8953	8872	8872
q10	3487	3606	3583	3583
q11	583	491	475	475
q12	829	593	591	591
q13	9166	3225	3194	3194
q14	313	293	289	289
q15	599	551	532	532
q16	506	467	484	467
q17	1818	1788	1783	1783
q18	8604	8031	8114	8031
q19	1780	1774	1768	1768
q20	2135	1897	1886	1886
q21	5887	5586	5631	5586
q22	1121	1057	1035	1035
Total cold run time: 69995 ms
Total hot run time: 61597 ms

@yiguolei
yiguoleiforce-pushed the fix-replace_empty-with-utf8 branch from d376e8b to fdcb4a4CompareSeptember 16, 2024 01:09
@yiguolei

yiguolei commented Sep 16, 2024

Copy link
Copy Markdown
Contributor

run buildall

@doris-robot

doris-robot commented Sep 16, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 37.32% (9578/25663)
Line Coverage: 28.70% (79155/275763)
Region Coverage: 28.18% (40977/145400)
Branch Coverage: 24.80% (20883/84194)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fdcb4a4cdd966be5233513838f587b79a2ac9d11_fdcb4a4cdd966be5233513838f587b79a2ac9d11/report/index.html

@doris-robot

doris-robot commented Sep 16, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 41029 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fdcb4a4cdd966be5233513838f587b79a2ac9d11, data reload: false
------ Round 1 ----------------------------------
q1	17576	7342	7247	7247
q2	2061	169	150	150
q3	10646	1109	1175	1109
q4	10542	723	723	723
q5	7752	3057	3033	3033
q6	227	147	145	145
q7	1001	611	606	606
q8	9421	2009	2037	2009
q9	6800	6417	6391	6391
q10	7013	2243	2252	2243
q11	441	242	244	242
q12	397	208	213	208
q13	17780	2979	2978	2978
q14	240	205	213	205
q15	575	521	519	519
q16	516	407	419	407
q17	970	811	832	811
q18	7253	6631	6693	6631
q19	1386	1113	902	902
q20	564	283	283	283
q21	4006	3242	3184	3184
q22	1125	1009	1003	1003
Total cold run time: 108292 ms
Total hot run time: 41029 ms
----- Round 2, with runtime_filter_mode=off -----
q1	7266	7354	7215	7215
q2	335	230	230	230
q3	2913	2944	2973	2944
q4	2021	1890	1906	1890
q5	5562	5595	5570	5570
q6	230	153	142	142
q7	2238	1790	1769	1769
q8	3310	3429	3387	3387
q9	8742	8806	8721	8721
q10	3509	3502	3447	3447
q11	604	502	490	490
q12	822	600	614	600
q13	10814	3176	3194	3176
q14	304	298	268	268
q15	587	527	525	525
q16	528	480	484	480
q17	1797	1589	1580	1580
q18	8158	7760	7765	7760
q19	1697	1637	1490	1490
q20	2134	1853	1836	1836
q21	5669	5184	5458	5184
q22	1174	1045	1014	1014
Total cold run time: 70414 ms
Total hot run time: 59718 ms

@doris-robot

doris-robot commented Sep 16, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 198753 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fdcb4a4cdd966be5233513838f587b79a2ac9d11, data reload: false
query1	1288	872	873	872
query2	6384	2062	1926	1926
query3	10794	3877	3974	3877
query4	64339	28239	23425	23425
query5	4924	477	465	465
query6	413	182	169	169
query7	5459	295	291	291
query8	312	214	218	214
query9	8340	2614	2603	2603
query10	445	298	276	276
query11	16672	15171	15819	15171
query12	164	109	102	102
query13	1467	433	410	410
query14	10835	7703	7815	7703
query15	215	185	183	183
query16	6857	533	439	439
query17	1205	626	574	574
query18	1592	311	317	311
query19	223	154	152	152
query20	130	110	104	104
query21	204	107	105	105
query22	4755	4520	4703	4520
query23	34807	33881	34244	33881
query24	6008	2908	2869	2869
query25	511	403	403	403
query26	650	168	157	157
query27	1675	284	289	284
query28	3995	2146	2100	2100
query29	670	432	442	432
query30	230	156	150	150
query31	951	760	797	760
query32	71	52	55	52
query33	430	300	288	288
query34	899	480	486	480
query35	809	738	751	738
query36	1065	958	940	940
query37	138	76	76	76
query38	3993	3982	3941	3941
query39	1444	1423	1397	1397
query40	212	97	96	96
query41	50	49	47	47
query42	115	96	99	96
query43	537	506	489	489
query44	1167	807	794	794
query45	199	166	163	163
query46	1143	803	773	773
query47	1862	1788	1793	1788
query48	457	370	360	360
query49	709	402	397	397
query50	835	392	491	392
query51	6933	6857	6892	6857
query52	102	87	88	87
query53	241	175	180	175
query54	574	450	446	446
query55	79	74	78	74
query56	267	255	248	248
query57	1189	1092	1106	1092
query58	219	225	236	225
query59	3196	3080	2902	2902
query60	275	263	264	263
query61	107	115	119	115
query62	757	650	664	650
query63	215	183	183	183
query64	1359	660	661	660
query65	3288	3222	3212	3212
query66	701	301	288	288
query67	16151	15517	15399	15399
query68	1302	867	842	842
query69	456	340	338	338
query70	1200	1197	1183	1183
query71	342	330	359	330
query72	6146	3368	3436	3368
query73	593	574	576	574
query74	9251	9085	8984	8984
query75	2974	2826	2932	2826
query76	1077	852	836	836
query77	376	355	347	347
query78	10205	9383	9281	9281
query79	917	881	882	881
query80	585	569	553	553
query81	450	240	238	238
query82	187	190	191	190
query83	158	158	153	153
query84	267	107	158	107
query85	662	363	357	357
query86	309	322	327	322
query87	4451	4336	4321	4321
query88	4202	4004	3975	3975
query89	369	355	358	355
query90	1444	304	298	298
query91	164	160	201	160
query92	72	68	70	68
query93	892	900	892	892
query94	554	352	377	352
query95	414	404	403	403
query96	473	476	473	473
query97	3121	3158	3123	3123
query98	235	236	238	236
query99	1440	1289	1276	1276
Total cold run time: 303590 ms
Total hot run time: 198753 ms

@doris-robot

doris-robot commented Sep 16, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.01 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fdcb4a4cdd966be5233513838f587b79a2ac9d11, data reload: false
query1	0.05	0.04	0.04
query2	0.07	0.03	0.02
query3	0.23	0.06	0.06
query4	1.65	0.10	0.10
query5	0.51	0.51	0.51
query6	1.13	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.56	0.49	0.50
query10	0.56	0.57	0.56
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.59
query14	1.44	1.40	1.44
query15	0.83	0.82	0.81
query16	0.38	0.37	0.38
query17	1.01	1.06	1.03
query18	0.20	0.19	0.19
query19	1.90	1.86	1.84
query20	0.01	0.01	0.01
query21	15.40	0.58	0.58
query22	2.30	2.69	1.51
query23	17.48	1.02	0.85
query24	2.95	0.67	0.12
query25	0.16	0.06	0.04
query26	0.38	0.14	0.13
query27	0.03	0.04	0.04
query28	11.98	1.08	1.08
query29	12.57	3.21	3.20
query30	0.24	0.05	0.06
query31	2.87	0.38	0.38
query32	3.29	0.47	0.46
query33	2.97	3.05	3.07
query34	16.88	4.41	4.39
query35	4.41	4.38	4.44
query36	0.67	0.47	0.49
query37	0.08	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.02	0.02
query40	0.15	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.02	0.02
Total cold run time: 106.53 s
Total hot run time: 30.01 s

HappenLee
HappenLee approved these changes Sep 18, 2024
View reviewed changes

@HappenLeeHappenLee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actionsBot commented Sep 18, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 18, 2024
@github-actions

github-actionsBot commented Sep 18, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@zhangstar333
zhangstar333 merged commit 2385734 into apache:masterSep 19, 2024
dataroaring pushed a commit that referenced this pull request Oct 9, 2024
@Mryange@dataroaring
[fix](function)fix error result in replace_empty with utf8 chars (#40869
42a44eb
)
## Proposed changes
```
mysql [(none)]>select replace_empty('你a好b世c界','','b');
+-------------------------------------------+
| replace_empty('你a好b世c界', '', 'b') |
+-------------------------------------------+
| b�b�b�bab�b�b�bbb�b�b�bcb�b�b�b |
+----------------------------
mysql [(none)]>select replace_empty('你a好b世c界','','b');
+-------------------------------------------+
| replace_empty('你a好b世c界', '', 'b') |
+-------------------------------------------+
| b你bab好bbb世bcb界b |
+-------------------------------------------+
```
<!--Describe your changes.-->
@gavinchougavinchou mentioned this pull request Nov 26, 2024
Closed
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@Mryange@HappenLee
[fix](function)fix error result in replace_empty with utf8 chars (apa…
e310cda
…che#40869)
## Proposed changes
```
mysql [(none)]>select replace_empty('你a好b世c界','','b');
+-------------------------------------------+
| replace_empty('你a好b世c界', '', 'b') |
+-------------------------------------------+
| b�b�b�bab�b�b�bbb�b�b�bcb�b�b�b |
+----------------------------
mysql [(none)]>select replace_empty('你a好b世c界','','b');
+-------------------------------------------+
| replace_empty('你a好b世c界', '', 'b') |
+-------------------------------------------+
| b你bab好bbb世bcb界b |
+-------------------------------------------+
```
<!--Describe your changes.-->
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HappenLeeHappenLeeHappenLee approved these changes
@zclllyybbzclllyybbzclllyybb approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/3.0.3-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@Mryange@doris-robot@yiguolei@HappenLee@zclllyybb@zhangstar333@dataroaring