[opt](variables) enlarge the default value of max_allowed_packet by morningman · Pull Request #38697 · apache/doris

morningman · 2024-08-01T08:24:56Z

From 1MB to 16MB, same as MySQL 8.4.
This is to avoid "Packet for query is too large" error
when querying wide table or table with wide column.

doris-robot · 2024-08-01T08:25:01Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

morningman · 2024-08-01T08:25:23Z

run buildall

doris-robot · 2024-08-01T09:03:34Z

TPC-H: Total hot run time: 41269 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 608310b277104f5ee420c5a34563c04b4eb4aad7, data reload: false
------ Round 1 ----------------------------------
q1	17623	4054	4054	4054
q2	2020	199	198	198
q3	10466	1250	1288	1250
q4	10155	771	906	771
q5	7641	2965	2931	2931
q6	220	137	135	135
q7	1029	604	597	597
q8	9425	1948	1897	1897
q9	8508	6571	6572	6571
q10	8754	3800	3859	3800
q11	425	244	243	243
q12	433	235	218	218
q13	17754	2922	2936	2922
q14	269	235	243	235
q15	514	466	470	466
q16	515	380	383	380
q17	969	912	876	876
q18	7892	7213	7272	7213
q19	1436	1206	1204	1204
q20	552	308	328	308
q21	5269	4744	4729	4729
q22	346	272	271	271
Total cold run time: 112215 ms
Total hot run time: 41269 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4042	4002	3979	3979
q2	328	222	223	222
q3	2991	2985	3057	2985
q4	1983	1982	2063	1982
q5	5486	5503	5439	5439
q6	216	130	145	130
q7	2110	1757	1767	1757
q8	3310	3387	3336	3336
q9	8684	8715	8679	8679
q10	3937	4036	3933	3933
q11	555	453	450	450
q12	749	611	609	609
q13	15384	3155	3143	3143
q14	306	277	272	272
q15	531	506	484	484
q16	476	428	419	419
q17	1737	1738	1733	1733
q18	8056	7644	7649	7644
q19	1758	1710	1700	1700
q20	2051	1846	1904	1846
q21	5753	5451	5529	5451
q22	515	451	458	451
Total cold run time: 70958 ms
Total hot run time: 56644 ms

doris-robot · 2024-08-01T09:14:13Z

TPC-DS: Total hot run time: 169771 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 608310b277104f5ee420c5a34563c04b4eb4aad7, data reload: false
query1	919	370	377	370
query2	6502	1771	1736	1736
query3	6645	221	230	221
query4	20251	17870	17677	17677
query5	3605	501	503	501
query6	302	186	166	166
query7	4603	297	288	288
query8	248	211	193	193
query9	8503	2358	2350	2350
query10	436	286	262	262
query11	10474	10030	10070	10030
query12	119	88	92	88
query13	1647	381	366	366
query14	9491	6847	6841	6841
query15	204	160	162	160
query16	6841	496	430	430
query17	912	556	531	531
query18	1898	271	280	271
query19	192	137	135	135
query20	91	84	86	84
query21	200	102	103	102
query22	4194	4034	4024	4024
query23	34079	33811	33622	33622
query24	10416	3069	3061	3061
query25	659	412	402	402
query26	1772	158	154	154
query27	3023	299	288	288
query28	7552	2005	1987	1987
query29	1266	429	437	429
query30	238	154	152	152
query31	965	780	785	780
query32	102	56	54	54
query33	669	319	332	319
query34	941	506	492	492
query35	867	752	775	752
query36	1053	874	864	864
query37	296	82	80	80
query38	2855	2781	2772	2772
query39	905	819	799	799
query40	255	109	109	109
query41	46	45	44	44
query42	120	100	98	98
query43	521	423	436	423
query44	1160	729	744	729
query45	205	174	183	174
query46	1078	819	800	800
query47	1783	1703	1720	1703
query48	353	287	282	282
query49	917	410	432	410
query50	900	441	441	441
query51	6785	6776	6774	6774
query52	97	93	86	86
query53	263	183	190	183
query54	619	450	449	449
query55	77	75	74	74
query56	273	249	256	249
query57	1144	1045	1020	1020
query58	276	265	271	265
query59	2730	2489	2352	2352
query60	297	262	270	262
query61	96	91	91	91
query62	896	651	650	650
query63	209	183	178	178
query64	5548	1948	1870	1870
query65	3203	3092	3096	3092
query66	1289	335	336	335
query67	15008	14850	14721	14721
query68	4277	574	582	574
query69	654	357	299	299
query70	1077	1076	1107	1076
query71	445	281	276	276
query72	7096	2720	2481	2481
query73	788	332	335	332
query74	6012	5617	5624	5617
query75	3404	2701	2725	2701
query76	2361	1223	1265	1223
query77	535	308	301	301
query78	9505	8949	8931	8931
query79	1265	532	548	532
query80	871	499	547	499
query81	573	221	234	221
query82	947	131	131	131
query83	251	165	183	165
query84	258	78	79	78
query85	1089	305	304	304
query86	398	305	293	293
query87	3267	3087	3073	3073
query88	2940	2411	2406	2406
query89	397	295	285	285
query90	1672	194	197	194
query91	126	100	100	100
query92	61	49	54	49
query93	1371	618	602	602
query94	759	307	266	266
query95	379	272	266	266
query96	605	280	285	280
query97	3227	3082	3052	3052
query98	216	195	193	193
query99	1597	1276	1274	1274
Total cold run time: 260506 ms
Total hot run time: 169771 ms

doris-robot · 2024-08-01T09:19:42Z

ClickBench: Total hot run time: 29.95 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 608310b277104f5ee420c5a34563c04b4eb4aad7, data reload: false
query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.69	0.07	0.07
query5	0.49	0.48	0.48
query6	1.14	0.72	0.71
query7	0.02	0.02	0.01
query8	0.05	0.04	0.03
query9	0.57	0.51	0.51
query10	0.55	0.57	0.56
query11	0.16	0.12	0.12
query12	0.15	0.13	0.12
query13	0.60	0.60	0.60
query14	0.76	0.78	0.78
query15	0.90	0.86	0.85
query16	0.35	0.35	0.36
query17	0.98	0.99	1.01
query18	0.22	0.20	0.22
query19	1.91	1.74	1.73
query20	0.01	0.01	0.01
query21	15.41	0.78	0.65
query22	4.25	7.41	1.31
query23	17.96	1.44	1.32
query24	2.26	0.22	0.22
query25	0.18	0.08	0.08
query26	0.32	0.22	0.21
query27	0.46	0.22	0.23
query28	13.17	0.99	0.97
query29	12.54	3.32	3.29
query30	0.25	0.06	0.05
query31	2.87	0.40	0.41
query32	3.24	0.48	0.48
query33	2.95	2.95	2.95
query34	15.41	4.27	4.22
query35	4.30	4.27	4.29
query36	0.68	0.49	0.49
query37	0.20	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.16	0.12	0.13
query41	0.09	0.04	0.04
query42	0.06	0.04	0.05
query43	0.05	0.04	0.05
Total cold run time: 107.89 s
Total hot run time: 29.95 s

morningman · 2024-08-02T04:04:57Z

run buildall

doris-robot · 2024-08-02T04:58:07Z

TPC-H: Total hot run time: 41799 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a45adcea30c99b3ed8c93812abe4a5bf9a2986b1, data reload: false
------ Round 1 ----------------------------------
q1	17616	4143	4151	4143
q2	2019	199	201	199
q3	10467	1293	1376	1293
q4	10169	849	971	849
q5	7633	2946	2974	2946
q6	221	136	143	136
q7	1076	614	610	610
q8	9437	1894	1961	1894
q9	8474	6672	6640	6640
q10	8769	3859	3856	3856
q11	436	248	255	248
q12	418	227	223	223
q13	17758	2940	2966	2940
q14	268	244	245	244
q15	527	491	487	487
q16	527	402	389	389
q17	968	920	898	898
q18	7984	7228	7246	7228
q19	1406	1244	1227	1227
q20	564	336	332	332
q21	5312	4790	4737	4737
q22	351	285	280	280
Total cold run time: 112400 ms
Total hot run time: 41799 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4085	4179	4027	4027
q2	336	230	216	216
q3	3018	2986	3183	2986
q4	2006	2014	1997	1997
q5	5583	5515	5463	5463
q6	233	135	130	130
q7	2116	1769	1837	1769
q8	3312	3375	3309	3309
q9	8621	8688	8743	8688
q10	3971	4030	3972	3972
q11	551	444	441	441
q12	760	581	652	581
q13	16295	3111	3122	3111
q14	308	269	283	269
q15	544	493	473	473
q16	485	413	407	407
q17	1769	1700	1717	1700
q18	8120	7750	7780	7750
q19	1730	1711	1737	1711
q20	2036	1864	1862	1862
q21	5680	5570	5392	5392
q22	518	443	466	443
Total cold run time: 72077 ms
Total hot run time: 56697 ms

doris-robot · 2024-08-02T05:08:39Z

TPC-DS: Total hot run time: 169346 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a45adcea30c99b3ed8c93812abe4a5bf9a2986b1, data reload: false
query1	911	383	369	369
query2	6464	1706	1696	1696
query3	6645	211	217	211
query4	19863	17485	17343	17343
query5	3635	505	527	505
query6	297	184	177	177
query7	4856	309	300	300
query8	265	198	200	198
query9	8512	2352	2344	2344
query10	424	287	265	265
query11	10558	9985	10022	9985
query12	122	84	86	84
query13	1639	393	386	386
query14	9684	6730	7347	6730
query15	197	162	168	162
query16	6803	416	424	416
query17	929	559	543	543
query18	1872	282	279	279
query19	189	142	142	142
query20	91	103	82	82
query21	200	98	98	98
query22	4084	4144	4151	4144
query23	33844	33514	33563	33514
query24	9305	3117	3070	3070
query25	649	386	406	386
query26	864	148	148	148
query27	2302	283	284	283
query28	6506	2000	1981	1981
query29	964	439	438	438
query30	231	160	156	156
query31	947	773	767	767
query32	104	61	56	56
query33	693	309	325	309
query34	901	508	486	486
query35	849	754	775	754
query36	1080	869	886	869
query37	144	80	82	80
query38	3006	2852	2896	2852
query39	891	810	832	810
query40	200	112	117	112
query41	46	44	45	44
query42	116	98	93	93
query43	468	412	425	412
query44	1168	734	738	734
query45	212	179	177	177
query46	1082	819	790	790
query47	1839	1745	1727	1727
query48	364	295	296	295
query49	841	427	424	424
query50	893	436	425	425
query51	6713	6680	6559	6559
query52	107	94	92	92
query53	254	180	187	180
query54	605	461	450	450
query55	77	78	75	75
query56	276	259	258	258
query57	1105	1065	1072	1065
query58	269	279	277	277
query59	2571	2477	2392	2392
query60	284	273	274	273
query61	98	97	95	95
query62	867	656	661	656
query63	217	184	176	176
query64	4434	1896	1866	1866
query65	3155	3083	3096	3083
query66	834	337	334	334
query67	15428	14779	14754	14754
query68	4332	551	567	551
query69	439	295	283	283
query70	1122	1060	1078	1060
query71	374	280	278	278
query72	7010	2711	2491	2491
query73	756	331	325	325
query74	6085	5621	5667	5621
query75	3328	2725	2731	2725
query76	2147	1218	1252	1218
query77	422	300	297	297
query78	9349	8982	8891	8891
query79	1933	538	523	523
query80	1194	497	505	497
query81	558	217	230	217
query82	1053	136	133	133
query83	230	169	166	166
query84	265	79	83	79
query85	1185	314	305	305
query86	469	301	302	301
query87	3280	3177	3101	3101
query88	2965	2519	2474	2474
query89	384	286	298	286
query90	1675	188	194	188
query91	130	101	102	101
query92	57	51	52	51
query93	1728	616	605	605
query94	824	275	296	275
query95	374	263	261	261
query96	609	283	285	283
query97	3269	3052	3041	3041
query98	213	196	203	196
query99	1675	1298	1287	1287
Total cold run time: 255477 ms
Total hot run time: 169346 ms

doris-robot · 2024-08-02T05:14:09Z

ClickBench: Total hot run time: 29.85 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a45adcea30c99b3ed8c93812abe4a5bf9a2986b1, data reload: false
query1	0.04	0.05	0.04
query2	0.08	0.04	0.04
query3	0.22	0.04	0.04
query4	1.69	0.07	0.07
query5	0.49	0.48	0.49
query6	1.13	0.72	0.71
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.58	0.51	0.51
query10	0.57	0.58	0.56
query11	0.16	0.11	0.11
query12	0.14	0.13	0.12
query13	0.61	0.59	0.61
query14	0.77	0.79	0.80
query15	0.90	0.86	0.87
query16	0.36	0.35	0.34
query17	0.97	0.99	1.02
query18	0.26	0.21	0.22
query19	1.83	1.78	1.77
query20	0.01	0.01	0.01
query21	15.40	0.77	0.67
query22	3.63	8.96	1.04
query23	17.71	1.38	1.31
query24	2.30	0.24	0.22
query25	0.18	0.08	0.08
query26	0.31	0.21	0.21
query27	0.45	0.23	0.23
query28	13.19	0.99	0.98
query29	12.57	3.28	3.29
query30	0.26	0.06	0.06
query31	2.86	0.40	0.40
query32	3.24	0.48	0.48
query33	2.91	2.93	2.98
query34	15.43	4.28	4.27
query35	4.29	4.33	4.36
query36	0.68	0.48	0.47
query37	0.18	0.16	0.16
query38	0.17	0.16	0.14
query39	0.04	0.03	0.04
query40	0.17	0.14	0.15
query41	0.11	0.06	0.05
query42	0.05	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 107.06 s
Total hot run time: 29.85 s

dataroaring

LGTM

github-actions · 2024-08-02T10:54:13Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-08-02T10:54:15Z

PR approved by anyone and no changes requested.

) From 1MB to 16MB, same as MySQL 8.4. This is to avoid "Packet for query is too large" error when querying wide table or table with wide column.

…che#38697) From 1MB to 16MB, same as MySQL 8.4. This is to avoid "Packet for query is too large" error when querying wide table or table with wide column.

) (#39626) bp #38697

…che#38697) From 1MB to 16MB, same as MySQL 8.4. This is to avoid "Packet for query is too large" error when querying wide table or table with wide column.

morningman added kind/behavior-changed dev/2.1.x labels Aug 1, 2024

github-actionsBot added the doing label Aug 1, 2024

[opt](variables) enlarge the default value of max_allowed_packet
a45adce

morningman force-pushed the large_max_packet branch from 608310b to a45adceCompare August 2, 2024 04:04

dataroaring approved these changes Aug 2, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Aug 2, 2024

github-actionsBot added the reviewed label Aug 2, 2024

yiguolei approved these changes Aug 2, 2024
View reviewed changes

yiguolei merged commit 024a4f0 into apache:masterAug 2, 2024

dataroaring added the dev/3.0.2-merged label Aug 12, 2024

morningman mentioned this pull request Aug 20, 2024
[opt](variables) enlarge the default value of max_allowed_packet (#38697) #39626
Merged

morningman added a commit that referenced this pull request Aug 20, 2024
[opt](variables) enlarge the default value of max_allowed_packet (#38697
a5daa3e
) (#39626) bp #38697

morningman added dev/2.1.6-merged and removed dev/2.1.x labels Aug 20, 2024

yiguolei mentioned this pull request Sep 5, 2024
Release Note 2.1.6 #40443
Closed
3 tasks

gavinchou mentioned this pull request Oct 13, 2024
Release Note 3.0.2 #41558
Closed

Navigation Menu

Uh oh!

[opt](variables) enlarge the default value of max_allowed_packet - #38697

[opt](variables) enlarge the default value of max_allowed_packet#38697
yiguolei merged 1 commit into
apache:masterfrom
morningman:large_max_packet

morningman commented Aug 1, 2024

Uh oh!

doris-robot commented Aug 1, 2024

Uh oh!

morningman commented Aug 1, 2024

Uh oh!

doris-robot commented Aug 1, 2024

Uh oh!

doris-robot commented Aug 1, 2024

Uh oh!

doris-robot commented Aug 1, 2024

Uh oh!

morningman commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

dataroaring left a comment

Uh oh!

github-actionsBot commented Aug 2, 2024

Uh oh!

github-actionsBot commented Aug 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

morningman commented Aug 1, 2024

Uh oh!

doris-robot commented Aug 1, 2024

Uh oh!

morningman commented Aug 1, 2024

Uh oh!

doris-robot commented Aug 1, 2024

Uh oh!

doris-robot commented Aug 1, 2024

Uh oh!

doris-robot commented Aug 1, 2024

Uh oh!

morningman commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

doris-robot commented Aug 2, 2024

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Aug 2, 2024

Uh oh!

github-actionsBot commented Aug 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants