[fix](nestedtype)fix nested type with is_exclusive by amorynan · Pull Request #40398 · apache/doris

amorynan · 2024-09-05T02:34:26Z

Proposed changes

is_exclusive in column_array/map/struct has wrong semantic , we should make sure it's nested column is also is_exclusive which can make behavior right in some operator like join.
Issue Number: close #xxx

doris-robot · 2024-09-05T02:34:31Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

amorynan · 2024-09-05T02:34:36Z

run buildall

github-actions · 2024-09-05T02:35:10Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/suites/datatype_p0/nested_types/query/sql/nested_with_join.sql

Consider using git-lfs to manage large files.

github-actions · 2024-09-05T02:41:06Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-09-05T02:53:54Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/suites/datatype_p0/nested_types/query/sql/nested_with_join.sql

Consider using git-lfs to manage large files.

github-actions · 2024-09-05T02:54:34Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-09-05T02:54:36Z

PR approved by anyone and no changes requested.

amorynan · 2024-09-05T02:56:46Z

run buildall

github-actions · 2024-09-05T03:00:02Z

clang-tidy review says "All clean, LGTM! 👍"

github-actions · 2024-09-05T03:13:24Z

Possible file(s) that should be tracked in LFS detected: 🚨

The following file(s) exceeds the file size limit: 1048576 bytes, as set in the .yml configuration files:

regression-test/suites/datatype_p0/nested_types/query/sql/nested_with_join.sql

Consider using git-lfs to manage large files.

amorynan · 2024-09-05T03:17:46Z

run buildall

github-actions · 2024-09-05T03:19:28Z

clang-tidy review says "All clean, LGTM! 👍"

eldenmoon

LGTM

github-actions · 2024-09-05T03:21:14Z

PR approved by at least one committer and no changes requested.

doris-robot · 2024-09-05T03:55:31Z

TPC-H: Total hot run time: 38388 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 271340573c1a2565363112b2bf0e5cf112a783ae, data reload: false
------ Round 1 ----------------------------------
q1	18424	4553	4390	4390
q2	3000	183	180	180
q3	10833	1240	1147	1147
q4	10479	788	756	756
q5	8124	2901	2875	2875
q6	227	140	139	139
q7	962	636	598	598
q8	9311	2075	2038	2038
q9	7201	6574	6527	6527
q10	7007	2255	2205	2205
q11	478	241	245	241
q12	390	218	220	218
q13	18876	3061	3071	3061
q14	271	235	247	235
q15	533	499	481	481
q16	584	502	511	502
q17	975	741	657	657
q18	7433	6892	6832	6832
q19	1390	1014	1072	1014
q20	697	339	329	329
q21	3953	2926	2941	2926
q22	1128	1038	1037	1037
Total cold run time: 112276 ms
Total hot run time: 38388 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4332	4305	4286	4286
q2	388	267	266	266
q3	2908	2673	2712	2673
q4	1938	1651	1612	1612
q5	5405	5393	5386	5386
q6	223	128	128	128
q7	2127	1746	1771	1746
q8	3199	3385	3332	3332
q9	8403	8415	8389	8389
q10	3457	3201	3161	3161
q11	585	501	507	501
q12	765	609	626	609
q13	10870	3063	3065	3063
q14	303	281	282	281
q15	536	486	477	477
q16	612	562	544	544
q17	1805	1510	1480	1480
q18	7788	7487	7438	7438
q19	1666	1581	1607	1581
q20	2060	1820	1841	1820
q21	5475	5097	5213	5097
q22	1118	1070	1025	1025
Total cold run time: 65963 ms
Total hot run time: 54895 ms

doris-robot · 2024-09-05T04:00:06Z

TeamCity be ut coverage result:
Function Coverage: 36.83% (9393/25502)
Line Coverage: 28.27% (77448/273946)
Region Coverage: 27.67% (39966/144460)
Branch Coverage: 24.31% (20339/83666)
Coverage Report: http://coverage.selectdb-in.cc/coverage/271340573c1a2565363112b2bf0e5cf112a783ae_271340573c1a2565363112b2bf0e5cf112a783ae/report/index.html

doris-robot · 2024-09-05T04:07:20Z

TPC-DS: Total hot run time: 188383 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 271340573c1a2565363112b2bf0e5cf112a783ae, data reload: false
query1	915	372	358	358
query2	6480	1935	1931	1931
query3	6651	203	223	203
query4	31376	23070	23078	23070
query5	4188	485	490	485
query6	261	171	178	171
query7	4591	285	294	285
query8	243	203	201	201
query9	8568	2509	2502	2502
query10	435	270	261	261
query11	17936	15029	14996	14996
query12	146	97	97	97
query13	1626	392	372	372
query14	9846	7195	7291	7195
query15	240	170	162	162
query16	8061	429	446	429
query17	1569	566	547	547
query18	2110	282	284	282
query19	191	141	138	138
query20	115	105	109	105
query21	212	102	101	101
query22	4544	4350	4272	4272
query23	34032	33503	33595	33503
query24	11230	2876	2783	2783
query25	636	415	403	403
query26	1206	159	160	159
query27	2360	283	286	283
query28	7202	2075	2075	2075
query29	844	438	419	419
query30	307	162	154	154
query31	986	767	779	767
query32	96	57	56	56
query33	769	295	293	293
query34	985	486	485	485
query35	852	741	731	731
query36	1109	941	950	941
query37	167	105	94	94
query38	3906	3877	3792	3792
query39	1430	1424	1393	1393
query40	205	117	117	117
query41	52	48	49	48
query42	119	95	100	95
query43	492	471	482	471
query44	1170	758	744	744
query45	202	169	177	169
query46	1105	766	748	748
query47	1925	1856	1845	1845
query48	390	303	307	303
query49	1111	444	459	444
query50	809	417	403	403
query51	7045	7014	6946	6946
query52	99	88	86	86
query53	256	188	180	180
query54	1028	469	479	469
query55	77	75	74	74
query56	310	274	267	267
query57	1207	1099	1086	1086
query58	252	235	234	234
query59	2903	2699	2870	2699
query60	308	280	272	272
query61	121	118	119	118
query62	847	671	653	653
query63	223	181	187	181
query64	5447	757	741	741
query65	3289	3151	3179	3151
query66	1443	343	359	343
query67	15834	15390	15672	15390
query68	4602	596	562	562
query69	419	287	275	275
query70	1109	1139	1164	1139
query71	341	271	287	271
query72	6746	3965	4028	3965
query73	743	329	329	329
query74	9126	8711	8824	8711
query75	3430	2730	2657	2657
query76	2456	1067	1050	1050
query77	512	306	313	306
query78	10311	9141	9141	9141
query79	2351	542	554	542
query80	1029	507	501	501
query81	586	235	232	232
query82	607	143	152	143
query83	232	154	150	150
query84	225	79	79	79
query85	1524	296	317	296
query86	483	301	269	269
query87	4373	4150	4194	4150
query88	3667	2382	2357	2357
query89	385	285	286	285
query90	1835	184	182	182
query91	121	99	100	99
query92	61	49	47	47
query93	2071	549	549	549
query94	888	287	279	279
query95	351	251	250	250
query96	591	271	261	261
query97	3279	3064	3080	3064
query98	222	205	202	202
query99	1668	1269	1286	1269
Total cold run time: 293034 ms
Total hot run time: 188383 ms

doris-robot · 2024-09-05T04:12:54Z

ClickBench: Total hot run time: 32.28 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 271340573c1a2565363112b2bf0e5cf112a783ae, data reload: false
query1	0.05	0.05	0.04
query2	0.08	0.04	0.03
query3	0.22	0.05	0.05
query4	1.68	0.08	0.08
query5	0.51	0.50	0.48
query6	1.13	0.73	0.73
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.54	0.49	0.48
query10	0.53	0.53	0.55
query11	0.15	0.11	0.11
query12	0.16	0.13	0.12
query13	0.60	0.59	0.60
query14	2.10	2.06	2.10
query15	0.90	0.83	0.82
query16	0.35	0.38	0.37
query17	1.02	1.05	1.03
query18	0.21	0.20	0.20
query19	1.83	1.72	1.77
query20	0.01	0.01	0.01
query21	15.40	0.66	0.65
query22	4.33	6.90	2.20
query23	18.28	1.35	1.33
query24	2.15	0.22	0.21
query25	0.14	0.07	0.08
query26	0.28	0.19	0.18
query27	0.07	0.08	0.07
query28	13.21	1.01	1.00
query29	12.59	3.31	3.27
query30	0.24	0.07	0.05
query31	2.87	0.42	0.40
query32	3.24	0.51	0.48
query33	2.99	3.00	3.07
query34	17.12	4.42	4.38
query35	4.50	4.44	4.43
query36	0.66	0.47	0.46
query37	0.19	0.16	0.16
query38	0.15	0.14	0.14
query39	0.05	0.04	0.04
query40	0.16	0.12	0.12
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.05
Total cold run time: 110.98 s
Total hot run time: 32.28 s

is_exclusive in column_array/map/struct has wrong semantic , we should make sure it's nested column is also is_exclusive which can make behavior right in some operator like join.

is_exclusive in column_array/map/struct has wrong semantic , we should make sure it's nested column is also is_exclusive which can make behavior right in some operator like join. ## Proposed changes backport: #40398 Issue Number: close #xxx

is_exclusive in column_array/map/struct has wrong semantic , we should make sure it's nested column is also is_exclusive which can make behavior right in some operator like join.

## Proposed changes backport: #40398 Issue Number: close #xxx

is_exclusive in column_array/map/struct has wrong semantic , we should make sure it's nested column is also is_exclusive which can make behavior right in some operator like join.

fix nested type with is_exclusive
fed4c5e

github-actionsBot added the lfs-detected! Warning Label for use when LFS is detected in the commits of a Pull Request label Sep 5, 2024

eldenmoon reviewed Sep 5, 2024
View reviewed changes

Comment threadbe/src/vec/columns/column_array.h

fix column object
c5f455a

Gabriel39 previously approved these changes Sep 5, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 5, 2024

github-actionsBot added the reviewed label Sep 5, 2024

amorynan requested a review from eldenmoon September 5, 2024 02:57

eldenmoon reviewed Sep 5, 2024
View reviewed changes

Comment threadbe/src/vec/columns/column_object.cpp Outdated

fixed
2713405

amorynan dismissed Gabriel39’s stale review via 2713405September 5, 2024 03:12

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Sep 5, 2024

eldenmoon approved these changes Sep 5, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 5, 2024

Gabriel39 approved these changes Sep 5, 2024
View reviewed changes

eldenmoon added dev/2.1.x dev/3.0.x labels Sep 5, 2024

eldenmoon merged commit 32557f8 into apache:masterSep 5, 2024

amorynan mentioned this pull request Sep 5, 2024
[fix](nestedtypes) fix nested type with is_exclusive #40434
Merged

yiguolei added dev/2.1.7-merged and removed dev/2.1.x labels Sep 19, 2024

xiaokang added the dev/2.0.x label Sep 23, 2024

amorynan mentioned this pull request Oct 10, 2024
[fix](is_exclusive) fix is_exlusive func for nested types #41646
Merged

dataroaring added dev/3.0.3-merged and removed dev/3.0.x labels Oct 12, 2024

eldenmoon pushed a commit that referenced this pull request Oct 15, 2024
[fix](is_exclusive) fix is_exlusive func for nested types (#41646)
3ab97fb
## Proposed changes backport: #40398 Issue Number: close #xxx

eldenmoon added dev/2.0.8-merged dev/2.0.16-merged-deprecated 2.0.16 is NOT and will NOT be released dev/2.1.7-merged and removed dev/2.0.x dev/2.1.7-merged dev/2.0.8-merged labels Oct 15, 2024

gavinchou mentioned this pull request Nov 26, 2024
Release Note 3.0.3 #44522
Closed

Uh oh!

Conversation

amorynan commented Sep 5, 2024

Proposed changes

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

amorynan commented Sep 5, 2024

Uh oh!

github-actionsBot commented Sep 5, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

github-actionsBot commented Sep 5, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

amorynan commented Sep 5, 2024

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

Uh oh!

github-actionsBot commented Sep 5, 2024

Possible file(s) that should be tracked in LFS detected: 🚨

Uh oh!

amorynan commented Sep 5, 2024

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

doris-robot commented Sep 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants