[Fix](inverted index) fix wrong no need read data when same column in inverted index and like function by airborne12 · Pull Request #36687 · apache/doris

airborne12 · 2024-06-21T11:35:13Z

Proposed changes

SQL like SELECT COUNT() FROM table WHERE a >= '2024-06-15 00:00:00' AND b LIKE '%xxx%' AND b MATCH 'xxx' for inverted index tables may cause incorrect optimization in the segment iterator, leading to the no-read-data optimization being applied incorrectly.

… inverted index and like function

doris-robot · 2024-06-21T11:35:18Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

airborne12 · 2024-06-21T11:36:10Z

run buildall

github-actions · 2024-06-21T11:40:56Z

clang-tidy review says "All clean, LGTM! 👍"

eldenmoon

lgtm

github-actions · 2024-06-21T11:47:52Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-06-21T11:47:54Z

PR approved by anyone and no changes requested.

doris-robot · 2024-06-21T11:53:40Z

TeamCity be ut coverage result:
Function Coverage: 36.48% (9009/24699)
Line Coverage: 28.02% (73888/263741)
Region Coverage: 27.49% (38373/139592)
Branch Coverage: 24.19% (19563/80860)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0ce7cf109ea1af987b3c0c7a49dda7ef6362903f_0ce7cf109ea1af987b3c0c7a49dda7ef6362903f/report/index.html

zzzxl1993

LGTM

doris-robot · 2024-06-21T12:09:19Z

TPC-H: Total hot run time: 39593 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0ce7cf109ea1af987b3c0c7a49dda7ef6362903f, data reload: false
------ Round 1 ----------------------------------
q1	17617	4370	4254	4254
q2	2038	188	201	188
q3	10451	1159	1119	1119
q4	10199	790	801	790
q5	7475	2625	2624	2624
q6	219	140	132	132
q7	949	624	591	591
q8	9227	2046	2028	2028
q9	8809	6455	6451	6451
q10	8977	3686	3685	3685
q11	444	241	241	241
q12	448	227	224	224
q13	17770	2949	3010	2949
q14	263	219	231	219
q15	529	499	486	486
q16	529	385	374	374
q17	950	685	622	622
q18	7991	7453	7357	7357
q19	5127	1469	1392	1392
q20	657	318	334	318
q21	5055	3202	3962	3202
q22	393	347	351	347
Total cold run time: 116117 ms
Total hot run time: 39593 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4360	4261	4202	4202
q2	376	266	258	258
q3	2974	2755	2943	2755
q4	2003	1731	1697	1697
q5	5593	5567	5509	5509
q6	220	129	131	129
q7	2204	1836	1924	1836
q8	3283	3375	3406	3375
q9	8632	8638	8810	8638
q10	4102	3999	3849	3849
q11	597	490	492	490
q12	823	646	659	646
q13	16997	3174	3155	3155
q14	304	267	278	267
q15	523	503	497	497
q16	480	430	425	425
q17	1782	1505	1480	1480
q18	8057	7925	7923	7923
q19	1816	1717	1603	1603
q20	3053	1917	1863	1863
q21	7829	5082	4733	4733
q22	869	562	564	562
Total cold run time: 76877 ms
Total hot run time: 55892 ms

doris-robot · 2024-06-21T12:20:18Z

TPC-DS: Total hot run time: 174602 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0ce7cf109ea1af987b3c0c7a49dda7ef6362903f, data reload: false
query1	946	380	401	380
query2	6470	2386	2371	2371
query3	6638	211	216	211
query4	19409	17374	17196	17196
query5	3563	478	503	478
query6	245	158	164	158
query7	4583	307	290	290
query8	312	301	289	289
query9	8700	2427	2395	2395
query10	559	303	296	296
query11	10559	10085	10153	10085
query12	121	87	85	85
query13	1652	372	363	363
query14	10351	7642	6966	6966
query15	228	196	232	196
query16	7647	271	256	256
query17	1563	537	515	515
query18	1888	272	279	272
query19	190	152	151	151
query20	90	90	85	85
query21	216	127	146	127
query22	4403	4117	4006	4006
query23	34015	33769	33671	33671
query24	11194	2969	2923	2923
query25	596	391	392	391
query26	703	157	155	155
query27	2295	331	329	329
query28	6249	2137	2117	2117
query29	870	619	632	619
query30	251	152	154	152
query31	946	784	745	745
query32	95	53	54	53
query33	725	315	305	305
query34	907	477	478	477
query35	733	664	635	635
query36	1186	961	988	961
query37	141	76	72	72
query38	2962	2831	2840	2831
query39	877	827	846	827
query40	205	126	134	126
query41	55	57	54	54
query42	111	108	116	108
query43	625	547	520	520
query44	1124	735	743	735
query45	194	157	165	157
query46	1055	699	715	699
query47	1901	1767	1780	1767
query48	382	295	297	295
query49	852	435	405	405
query50	760	396	388	388
query51	6965	6759	6804	6759
query52	101	98	99	98
query53	372	296	286	286
query54	889	449	454	449
query55	72	76	75	75
query56	276	252	261	252
query57	1124	1064	1061	1061
query58	249	265	231	231
query59	3446	3533	3451	3451
query60	301	289	283	283
query61	92	93	94	93
query62	607	430	439	430
query63	320	290	282	282
query64	8789	2288	1749	1749
query65	3218	3133	3078	3078
query66	750	322	328	322
query67	15342	14932	15049	14932
query68	4545	530	537	530
query69	591	479	416	416
query70	1200	1157	1078	1078
query71	441	279	277	277
query72	7549	5859	5758	5758
query73	747	323	317	317
query74	5926	5554	5468	5468
query75	3427	2634	2658	2634
query76	2603	995	854	854
query77	596	289	296	289
query78	10424	9877	9924	9877
query79	2454	511	510	510
query80	1930	550	471	471
query81	601	217	234	217
query82	844	107	100	100
query83	295	180	170	170
query84	258	86	82	82
query85	1050	289	269	269
query86	441	319	342	319
query87	3251	3105	3087	3087
query88	3694	2344	2334	2334
query89	476	379	384	379
query90	1621	194	182	182
query91	129	102	98	98
query92	55	50	48	48
query93	1823	507	500	500
query94	1048	190	182	182
query95	400	310	308	308
query96	576	269	269	269
query97	3245	3109	3031	3031
query98	241	207	203	203
query99	1272	841	826	826
Total cold run time: 269008 ms
Total hot run time: 174602 ms

doris-robot · 2024-06-21T12:25:50Z

ClickBench: Total hot run time: 30.77 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0ce7cf109ea1af987b3c0c7a49dda7ef6362903f, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.05
query3	0.22	0.05	0.05
query4	1.68	0.07	0.07
query5	0.50	0.50	0.50
query6	1.14	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.05	0.05
query9	0.55	0.49	0.50
query10	0.55	0.55	0.56
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.59	0.59	0.61
query14	0.78	0.78	0.80
query15	0.85	0.83	0.83
query16	0.36	0.37	0.37
query17	1.03	0.98	0.95
query18	0.24	0.24	0.23
query19	1.81	1.72	1.79
query20	0.01	0.01	0.01
query21	15.43	0.66	0.66
query22	4.14	7.48	1.94
query23	18.29	1.50	1.25
query24	2.13	0.24	0.26
query25	0.15	0.08	0.08
query26	0.27	0.18	0.18
query27	0.09	0.08	0.08
query28	13.23	1.03	1.01
query29	12.64	3.33	3.31
query30	0.26	0.07	0.07
query31	2.83	0.40	0.38
query32	3.29	0.48	0.47
query33	2.98	2.96	2.93
query34	17.21	4.39	4.39
query35	4.49	4.47	4.46
query36	0.66	0.48	0.50
query37	0.19	0.15	0.16
query38	0.17	0.16	0.15
query39	0.04	0.04	0.04
query40	0.17	0.15	0.14
query41	0.10	0.05	0.04
query42	0.05	0.05	0.04
query43	0.04	0.05	0.04
Total cold run time: 109.66 s
Total hot run time: 30.77 s

… inverted index and like function (#36687) ## Proposed changes SQL like SELECT COUNT() FROM table WHERE a >= '2024-06-15 00:00:00' AND b LIKE '%xxx%' AND b MATCH 'xxx' for inverted index tables may cause incorrect optimization in the segment iterator, leading to the no-read-data optimization being applied incorrectly.

… inverted index and like function (apache#36687) ## Proposed changes SQL like SELECT COUNT() FROM table WHERE a >= '2024-06-15 00:00:00' AND b LIKE '%xxx%' AND b MATCH 'xxx' for inverted index tables may cause incorrect optimization in the segment iterator, leading to the no-read-data optimization being applied incorrectly.

… inverted index and like function #36687 (#38581) cherry pick from #36687

… inverted index and like function (apache#36687) ## Proposed changes SQL like SELECT COUNT() FROM table WHERE a >= '2024-06-15 00:00:00' AND b LIKE '%xxx%' AND b MATCH 'xxx' for inverted index tables may cause incorrect optimization in the segment iterator, leading to the no-read-data optimization being applied incorrectly.

… inverted index and like function #36687 (#38651) cherry pick from #36687

… inverted index and like function apache#36687 (apache#38651) cherry pick from apache#36687

… inverted index and like function (apache#36687) ## Proposed changes SQL like SELECT COUNT() FROM table WHERE a >= '2024-06-15 00:00:00' AND b LIKE '%xxx%' AND b MATCH 'xxx' for inverted index tables may cause incorrect optimization in the segment iterator, leading to the no-read-data optimization being applied incorrectly.

[Fix](inverted index) fix wrong no need read data when same column in…
0ce7cf1
… inverted index and like function

eldenmoon approved these changes Jun 21, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 21, 2024

github-actionsBot added the reviewed label Jun 21, 2024

zzzxl1993 approved these changes Jun 21, 2024
View reviewed changes

airborne12 merged commit 3a9c29e into apache:masterJun 21, 2024

xiaokang added dev/2.0.x dev/2.1.x p0_w labels Jul 31, 2024

airborne12 mentioned this pull request Jul 31, 2024
[Fix](inverted index) fix wrong no need read data when same column in inverted index and like function #36687 #38581
Merged

airborne12 added a commit that referenced this pull request Jul 31, 2024
[Fix](inverted index) fix wrong no need read data when same column in…
7730aa2
… inverted index and like function #36687 (#38581) cherry pick from #36687

airborne12 added dev/2.1.6-merged and removed dev/2.1.x labels Jul 31, 2024

airborne12 mentioned this pull request Aug 1, 2024
[Fix](inverted index) fix wrong no need read data when same column in inverted index and like function #36687 #38651
Merged

airborne12 added a commit that referenced this pull request Aug 1, 2024
[Fix](inverted index) fix wrong no need read data when same column in…
22665e2
… inverted index and like function #36687 (#38651) cherry pick from #36687

airborne12 added dev/2.0.14-merged and removed dev/2.0.x labels Aug 1, 2024

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

yiguolei mentioned this pull request Sep 5, 2024
Release Note 2.1.6 #40443
Closed
3 tasks

mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
[Fix](inverted index) fix wrong no need read data when same column in…
19d34b5
… inverted index and like function apache#36687 (apache#38651) cherry pick from apache#36687

Navigation Menu

Uh oh!

[Fix](inverted index) fix wrong no need read data when same column in inverted index and like function - #36687

[Fix](inverted index) fix wrong no need read data when same column in inverted index and like function#36687
airborne12 merged 1 commit into
apache:masterfrom
airborne12:fix-need-read-data

airborne12 commented Jun 21, 2024

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

airborne12 commented Jun 21, 2024

Uh oh!

github-actionsBot commented Jun 21, 2024

Uh oh!

eldenmoon left a comment

Uh oh!

github-actionsBot commented Jun 21, 2024

Uh oh!

github-actionsBot commented Jun 21, 2024

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

zzzxl1993 left a comment

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

airborne12 commented Jun 21, 2024

Proposed changes

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

airborne12 commented Jun 21, 2024

Uh oh!

github-actionsBot commented Jun 21, 2024

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jun 21, 2024

Uh oh!

github-actionsBot commented Jun 21, 2024

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

zzzxl1993 left a comment

Choose a reason for hiding this comment

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

doris-robot commented Jun 21, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants