Skip to content

[fix](Row store) get the correct value of row_store_page_size after FE restarts - #38236

Merged
eldenmoon merged 39 commits into
apache:masterfrom
lxr599:page_size
Jul 25, 2024
Merged

[fix](Row store) get the correct value of row_store_page_size after FE restarts#38236
eldenmoon merged 39 commits into
apache:masterfrom
lxr599:page_size

Conversation

@lxr599

@lxr599lxr599 commented Jul 23, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

get the correct value of row_store_page_size after FE restarts

lxr599and others added 30 commits July 2, 2024 16:44
@lxr599
change default value of row_column_page_size to 16KB
ec59d28
@lxr599
allow to set row_column_page_size for tables
76a78da
@lxr599
[feature](Row store)allow to set row_column_page_size for tables
288bf5a
@lxr599
format codes
b544145
@lxr599
change default value of row_column_page_size to 16KB
2339bd1
@lxr599
allow to set row_column_page_size for tables
9a211b1
@lxr599
[feature](Row store)allow to set row_column_page_size for tables
9552bd5
@lxr599
format codes
8c4c0f7
@lxr599
rebase master
a9e584a
@lxr599
Merge branch 'master' into page_size
eedaa21
@lxr599
format codes
3f271b0
@lxr599
format fe codes
843d26d
@lxr599
Merge remote-tracking branch 'upstream/master' into page_size
2936a27
@lxr599
Merge branch 'master' into page_size
82a3e84
@lxr599
Merge branch 'master' into page_size
50dc477
@lxr599
display row_column_page_size only when row store enabled
fe22103
@lxr599
Merge branch 'master' into page_size
9706f9a
@lxr599
add test case for property row_column_page_size
9860a7f
@lxr599
Merge branch 'master' into page_size
29f3fd7
@lxr599
Merge branch 'apache:master' into page_size
78ba814
@lxr599
add more test cases
05374ba
@lxr599
Merge branch 'master' into page_size
a607007
@lxr599
change the property name
c32b788
@lxr599
change names of vars related to row_store_page_size
484160a
@lxr599
Merge remote-tracking branch 'upstream/master' into page_size
5c543f7
@lxr599
Merge branch 'master' into page_size
617dafd
@lxr599
Merge branch 'apache:master' into page_size
f7d118a
@lxr599
Merge branch 'apache:master' into page_size
e1b95e7
@lxr599
compatible with 'branch-3.0'
07beb0c
@lxr599
Merge branch 'apache:master' into page_size
fdb5b5a
@doris-robot

doris-robot commented Jul 24, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 174056 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 26e18ac623c12ffa189e17ac6897901ce8faa0c0, data reload: false
query1	908	368	378	368
query2	6432	1963	1848	1848
query3	6641	205	215	205
query4	26767	17694	17447	17447
query5	3811	491	503	491
query6	264	176	169	169
query7	4594	287	277	277
query8	227	190	184	184
query9	8534	2400	2390	2390
query10	432	318	280	280
query11	11798	10086	10112	10086
query12	111	81	80	80
query13	1635	362	357	357
query14	9445	7542	7698	7542
query15	217	175	173	173
query16	7738	499	473	473
query17	1364	538	535	535
query18	1907	276	272	272
query19	199	151	155	151
query20	92	86	82	82
query21	211	127	128	127
query22	4499	4310	3998	3998
query23	34172	33622	33669	33622
query24	10877	2938	2872	2872
query25	605	403	398	398
query26	705	149	148	148
query27	2252	276	289	276
query28	6193	2083	2055	2055
query29	867	634	633	633
query30	268	155	153	153
query31	986	779	770	770
query32	91	53	62	53
query33	640	356	347	347
query34	920	497	515	497
query35	865	733	763	733
query36	1141	978	948	948
query37	137	81	84	81
query38	2952	2830	2752	2752
query39	877	813	831	813
query40	200	121	117	117
query41	46	43	45	43
query42	112	99	97	97
query43	501	470	463	463
query44	1112	722	723	722
query45	190	159	160	159
query46	1101	728	756	728
query47	1844	1796	1798	1796
query48	353	291	296	291
query49	856	405	436	405
query50	772	381	389	381
query51	6776	6678	6763	6678
query52	98	95	91	91
query53	358	289	285	285
query54	861	456	445	445
query55	76	74	76	74
query56	285	265	268	265
query57	1146	1044	1044	1044
query58	237	252	273	252
query59	2938	2670	2559	2559
query60	322	273	265	265
query61	98	95	95	95
query62	798	640	651	640
query63	321	285	287	285
query64	9249	2324	1785	1785
query65	3167	3107	3109	3107
query66	740	321	319	319
query67	15583	15091	14853	14853
query68	4622	549	539	539
query69	730	426	349	349
query70	1154	1158	1061	1061
query71	411	278	276	276
query72	7692	5696	5484	5484
query73	775	342	333	333
query74	6432	6003	5721	5721
query75	3448	2693	2716	2693
query76	2684	953	933	933
query77	636	312	311	311
query78	9653	9202	10144	9202
query79	2993	517	536	517
query80	1360	464	467	464
query81	583	217	218	217
query82	736	137	132	132
query83	332	164	168	164
query84	271	86	84	84
query85	1666	321	300	300
query86	473	325	321	321
query87	3268	3106	3117	3106
query88	4060	2377	2385	2377
query89	472	395	382	382
query90	1729	193	196	193
query91	132	102	101	101
query92	58	48	49	48
query93	2387	512	519	512
query94	1132	278	304	278
query95	416	317	315	315
query96	600	274	276	274
query97	3170	3024	3002	3002
query98	230	199	205	199
query99	1637	1238	1270	1238
Total cold run time: 278456 ms
Total hot run time: 174056 ms

@doris-robot

doris-robot commented Jul 24, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.53 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 26e18ac623c12ffa189e17ac6897901ce8faa0c0, data reload: false
query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.22	0.04	0.05
query4	1.68	0.08	0.08
query5	0.50	0.49	0.48
query6	1.13	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.55	0.50	0.48
query10	0.54	0.54	0.54
query11	0.15	0.12	0.11
query12	0.14	0.13	0.12
query13	0.59	0.60	0.59
query14	0.75	0.77	0.78
query15	0.86	0.81	0.82
query16	0.36	0.36	0.37
query17	0.95	0.94	0.95
query18	0.23	0.23	0.22
query19	1.81	1.68	1.69
query20	0.01	0.01	0.01
query21	15.42	0.74	0.65
query22	4.66	6.96	1.99
query23	18.29	1.43	1.26
query24	2.09	0.22	0.22
query25	0.15	0.10	0.08
query26	0.29	0.21	0.21
query27	0.45	0.22	0.23
query28	13.32	1.02	1.00
query29	12.60	3.32	3.31
query30	0.25	0.05	0.06
query31	2.88	0.39	0.39
query32	3.26	0.49	0.46
query33	2.89	2.93	2.86
query34	16.96	4.37	4.33
query35	4.43	4.42	4.38
query36	0.65	0.50	0.46
query37	0.19	0.16	0.15
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.14	0.12	0.13
query41	0.09	0.05	0.04
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.97 s
Total hot run time: 30.53 s

@lxr599

lxr599 commented Jul 24, 2024

Copy link
Copy Markdown
ContributorAuthor

run p0

@lxr599

lxr599 commented Jul 24, 2024

Copy link
Copy Markdown
ContributorAuthor

run buildall

@github-actions

github-actionsBot commented Jul 24, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented Jul 24, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 39831 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 297dc779c2e1a2e002e0041c33a09a620fb9e692, data reload: false
------ Round 1 ----------------------------------
q1	17661	4543	4268	4268
q2	2026	193	184	184
q3	10519	1167	1148	1148
q4	10248	794	847	794
q5	7529	2707	2648	2648
q6	221	142	141	141
q7	958	602	598	598
q8	9219	2056	2045	2045
q9	8729	6516	6549	6516
q10	8709	3825	3764	3764
q11	475	233	240	233
q12	389	231	232	231
q13	17777	2987	2988	2987
q14	283	230	235	230
q15	518	502	479	479
q16	499	390	385	385
q17	958	707	750	707
q18	8204	7350	7314	7314
q19	8193	1409	1366	1366
q20	661	324	332	324
q21	5009	3189	3269	3189
q22	355	280	286	280
Total cold run time: 119140 ms
Total hot run time: 39831 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4397	4264	4204	4204
q2	364	264	260	260
q3	3021	2890	2979	2890
q4	1943	1699	1713	1699
q5	5532	5550	5444	5444
q6	220	137	136	136
q7	2286	1899	1879	1879
q8	3295	3433	3399	3399
q9	8828	8898	8792	8792
q10	4165	3676	3863	3676
q11	618	523	508	508
q12	803	666	636	636
q13	17191	3162	3226	3162
q14	316	294	287	287
q15	531	482	496	482
q16	482	467	440	440
q17	1795	1488	1494	1488
q18	8104	7953	7757	7757
q19	1763	1564	1521	1521
q20	3094	1889	1874	1874
q21	5101	4765	4746	4746
q22	678	537	489	489
Total cold run time: 74527 ms
Total hot run time: 55769 ms

@doris-robot

doris-robot commented Jul 24, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 174307 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 297dc779c2e1a2e002e0041c33a09a620fb9e692, data reload: false
query1	917	382	377	377
query2	6423	1937	1821	1821
query3	6640	209	216	209
query4	28374	17681	17390	17390
query5	3642	487	495	487
query6	281	177	171	171
query7	4611	295	290	290
query8	258	196	194	194
query9	8373	2405	2404	2404
query10	438	306	276	276
query11	11325	9979	10111	9979
query12	117	91	90	90
query13	1651	380	370	370
query14	10422	7984	7540	7540
query15	214	157	168	157
query16	7887	493	473	473
query17	1586	544	520	520
query18	2015	271	277	271
query19	193	147	151	147
query20	91	83	81	81
query21	198	123	129	123
query22	4338	4033	4105	4033
query23	34233	33657	33632	33632
query24	11057	2898	2889	2889
query25	620	408	401	401
query26	1223	161	153	153
query27	2409	286	280	280
query28	7197	2071	2049	2049
query29	881	651	606	606
query30	251	152	162	152
query31	952	747	767	747
query32	98	53	54	53
query33	761	330	338	330
query34	927	507	504	504
query35	832	774	761	761
query36	1133	990	981	981
query37	160	90	81	81
query38	2982	2922	2862	2862
query39	871	863	836	836
query40	200	122	118	118
query41	45	44	46	44
query42	120	96	96	96
query43	495	464	464	464
query44	1204	727	720	720
query45	206	175	175	175
query46	1088	731	753	731
query47	1837	1799	1778	1778
query48	374	290	294	290
query49	842	399	410	399
query50	774	391	399	391
query51	6779	6712	6721	6712
query52	106	96	89	89
query53	351	294	282	282
query54	918	453	451	451
query55	77	77	74	74
query56	302	269	284	269
query57	1137	1058	1042	1042
query58	246	247	269	247
query59	2752	2542	2562	2542
query60	309	268	269	268
query61	97	95	93	93
query62	821	625	659	625
query63	325	288	283	283
query64	9430	2257	1723	1723
query65	3162	3099	3100	3099
query66	747	328	337	328
query67	15397	14866	14882	14866
query68	4457	537	557	537
query69	463	341	339	339
query70	1134	1137	1169	1137
query71	422	279	285	279
query72	7106	6438	5733	5733
query73	752	328	330	328
query74	6098	5740	5752	5740
query75	3430	2699	2715	2699
query76	2717	927	936	927
query77	419	298	304	298
query78	9821	9141	9166	9141
query79	2488	532	523	523
query80	2186	471	471	471
query81	607	224	223	223
query82	996	140	136	136
query83	304	164	228	164
query84	264	87	95	87
query85	1204	310	298	298
query86	462	286	333	286
query87	3262	3130	3092	3092
query88	3385	2363	2366	2363
query89	489	385	378	378
query90	1735	196	193	193
query91	137	104	102	102
query92	67	51	49	49
query93	1909	522	519	519
query94	1101	305	288	288
query95	411	321	315	315
query96	590	271	274	271
query97	3196	3023	3098	3023
query98	225	198	198	198
query99	1568	1251	1274	1251
Total cold run time: 280127 ms
Total hot run time: 174307 ms

@doris-robot

doris-robot commented Jul 24, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.95 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 297dc779c2e1a2e002e0041c33a09a620fb9e692, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.05
query4	1.68	0.09	0.11
query5	0.52	0.47	0.47
query6	1.15	0.73	0.73
query7	0.02	0.02	0.01
query8	0.06	0.04	0.04
query9	0.55	0.48	0.48
query10	0.54	0.53	0.53
query11	0.16	0.11	0.12
query12	0.14	0.12	0.13
query13	0.60	0.59	0.59
query14	0.77	0.79	0.78
query15	0.86	0.82	0.82
query16	0.37	0.37	0.36
query17	0.95	0.95	0.97
query18	0.24	0.22	0.21
query19	1.81	1.71	1.74
query20	0.01	0.02	0.01
query21	15.47	0.78	0.64
query22	4.22	6.46	2.28
query23	18.27	1.38	1.28
query24	2.18	0.24	0.22
query25	0.15	0.08	0.08
query26	0.30	0.21	0.22
query27	0.46	0.24	0.23
query28	13.13	1.04	1.02
query29	12.63	3.34	3.32
query30	0.25	0.06	0.05
query31	2.88	0.39	0.40
query32	3.26	0.50	0.47
query33	2.93	2.86	2.91
query34	16.96	4.33	4.36
query35	4.39	4.37	4.43
query36	0.66	0.50	0.47
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.16	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.67 s
Total hot run time: 30.95 s

eldenmoon
eldenmoon approved these changes Jul 24, 2024
View reviewed changes

@eldenmooneldenmoon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actionsBot commented Jul 24, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actionsgithub-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 24, 2024
@github-actions

github-actionsBot commented Jul 24, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

xiaokang
xiaokang approved these changes Jul 25, 2024
View reviewed changes

@xiaokangxiaokang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon
eldenmoon merged commit 7795533 into apache:masterJul 25, 2024
eldenmoon pushed a commit that referenced this pull request Jul 25, 2024
@lxr599
[fix](Row store) get the correct value of row_store_page_size after F…
ccd0b93
…E restarts (#38240)
pick #38236
@lxr599lxr599 mentioned this pull request Aug 6, 2024
Merged
xiaokang pushed a commit that referenced this pull request Aug 7, 2024
@lxr599
[Optimize](Row store) pick#37145,#38236(#38932)
2543b56
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@lxr599@HappenLee
[fix](Row store) get the correct value of row_store_page_size after…
8d24214
… FE restarts (apache#38236)
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xiaokangxiaokangxiaokang approved these changes
@eldenmooneldenmooneldenmoon approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.6-mergeddev/3.0.1-mergedmeta-changereviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@lxr599@doris-robot@xiaokang@eldenmoon