Skip to content

[fix](fe) Reject JSONB and variant distribution columns - #63211

Merged
starocean999 merged 1 commit into
apache:masterfrom
CalvinKirs:tbl_check_jsonb
May 15, 2026
Merged

[fix](fe) Reject JSONB and variant distribution columns#63211
starocean999 merged 1 commit into
apache:masterfrom
CalvinKirs:tbl_check_jsonb

Conversation

@CalvinKirs

@CalvinKirsCalvinKirs commented May 13, 2026

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Hash distribution column validation missed JSONB and VARIANT columns, allowing CREATE TABLE statements such as DISTRIBUTED BY HASH(jsonb_col) even though those types are rejected in key and partition paths. This change centralizes hash distribution type validation and reuses it from both catalog and Nereids create-table paths.

Release note

Reject JSONB and VARIANT columns as hash distribution columns during CREATE TABLE analysis.

Check List (For Author)

  • Test: Unit Test and Regression test attempt
    • Unit Test: env FE_UT_PARALLEL=0 ./run-fe-ut.sh --run org.apache.doris.nereids.trees.plans.commands.info.DistributionDescriptorTest
    • Regression test: ./run-regression-test.sh --run -d jsonb_p0 -s test_jsonb_distribution_column (attempted against the existing 127.0.0.1:9030 cluster; it failed because that cluster was not rebuilt from this branch and still accepted HASH(JSONB), confirmed by direct mysql SHOW CREATE TABLE)
  • Behavior changed: Yes. CREATE TABLE now rejects JSONB and VARIANT hash distribution columns.
  • Does this need documentation: No

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@CalvinKirs

CalvinKirs commented May 13, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29814 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7dcd77a14168da867b16e92b795df5d1db97c771, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17595	4006	4114	4006
q2	q3	10700	871	625	625
q4	4663	469	346	346
q5	7452	1347	1132	1132
q6	194	183	147	147
q7	906	960	747	747
q8	9307	1407	1235	1235
q9	5801	5345	5303	5303
q10	6242	2087	1788	1788
q11	461	273	257	257
q12	629	434	309	309
q13	18079	3434	2731	2731
q14	294	282	261	261
q15	q16	912	879	786	786
q17	911	991	842	842
q18	6547	5721	5555	5555
q19	1182	1259	1121	1121
q20	518	403	262	262
q21	4631	2490	2011	2011
q22	467	421	350	350
Total cold run time: 97491 ms
Total hot run time: 29814 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4918	4733	4865	4733
q2	q3	4653	4789	4239	4239
q4	2162	2249	1437	1437
q5	4993	5102	5320	5102
q6	225	192	146	146
q7	2117	1855	1616	1616
q8	3339	3083	3101	3083
q9	8376	8364	8445	8364
q10	4464	4765	4261	4261
q11	621	422	390	390
q12	706	744	536	536
q13	3329	3753	2998	2998
q14	298	303	275	275
q15	q16	754	953	676	676
q17	1365	1334	1281	1281
q18	7959	7158	7199	7158
q19	1171	1163	1190	1163
q20	2233	2230	1944	1944
q21	6276	5555	5052	5052
q22	566	521	441	441
Total cold run time: 60525 ms
Total hot run time: 54895 ms

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 169956 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7dcd77a14168da867b16e92b795df5d1db97c771, data reload: false
query5	4338	677	503	503
query6	316	221	208	208
query7	4265	551	298	298
query8	321	228	225	225
query9	8833	4047	3993	3993
query10	453	356	305	305
query11	5773	2407	2234	2234
query12	177	135	127	127
query13	1309	571	418	418
query14	5951	5326	5029	5029
query14_1	4331	4315	4308	4308
query15	213	201	182	182
query16	1041	476	438	438
query17	1122	769	622	622
query18	2509	478	352	352
query19	217	202	159	159
query20	136	130	129	129
query21	214	142	119	119
query22	13664	13535	13442	13442
query23	17073	16406	16049	16049
query23_1	16113	16036	16179	16036
query24	7523	1775	1358	1358
query24_1	1354	1304	1340	1304
query25	553	495	463	463
query26	1283	310	169	169
query27	2730	595	343	343
query28	4456	2004	1959	1959
query29	987	635	500	500
query30	303	238	200	200
query31	1120	1056	945	945
query32	86	73	72	72
query33	528	355	298	298
query34	1160	1156	669	669
query35	766	787	669	669
query36	1314	1321	1177	1177
query37	159	105	89	89
query38	3216	3138	3064	3064
query39	932	926	892	892
query39_1	902	868	881	868
query40	237	160	142	142
query41	72	69	68	68
query42	116	112	112	112
query43	333	332	291	291
query44 query45	215	203	199	199
query46	1094	1243	733	733
query47	2350	2317	2167	2167
query48	406	438	297	297
query49	671	557	439	439
query50	745	289	227	227
query51	4318	4288	4190	4190
query52	105	107	96	96
query53	251	283	215	215
query54	329	292	287	287
query55	98	97	87	87
query56	320	332	305	305
query57	1388	1395	1303	1303
query58	304	291	288	288
query59	1575	1640	1383	1383
query60	359	348	334	334
query61	182	179	178	178
query62	675	627	533	533
query63	239	196	211	196
query64	2505	877	720	720
query65 query66	1749	545	408	408
query67	29989	29954	29832	29832
query68 query69	480	347	318	318
query70	1057	962	985	962
query71	317	282	266	266
query72	3134	2651	2397	2397
query73	817	783	400	400
query74	5070	4944	4747	4747
query75	2782	2643	2334	2334
query76	2291	1117	767	767
query77	416	433	348	348
query78	12842	12918	12352	12352
query79	1529	967	759	759
query80	1341	572	498	498
query81	515	279	239	239
query82	1206	160	119	119
query83	387	278	259	259
query84	255	146	112	112
query85	906	502	452	452
query86	436	355	330	330
query87	3479	3365	3244	3244
query88	3562	2712	2686	2686
query89	451	383	339	339
query90	1920	177	181	177
query91	177	163	145	145
query92	82	77	72	72
query93	990	966	569	569
query94	713	341	267	267
query95	664	474	358	358
query96	1009	813	355	355
query97	2696	2684	2543	2543
query98	238	229	226	226
query99	1119	1097	966	966
Total cold run time: 254264 ms
Total hot run time: 169956 ms

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 68.42% (13/19) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 65.38% (17/26) 🎉
Increment coverage report
Complete coverage report

@CalvinKirs

CalvinKirs commented May 13, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

1 similar comment
@CalvinKirs

CalvinKirs commented May 13, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

starocean999
starocean999 previously approved these changes May 13, 2026
View reviewed changes
@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label May 13, 2026
@github-actions

github-actionsBot commented May 13, 2026

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented May 13, 2026

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29479 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3636d6136379dbf64a9c53a1ce970959e2c6acb2, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17633	3904	3901	3901
q2	q3	10697	867	613	613
q4	4668	475	351	351
q5	7435	1364	1143	1143
q6	197	177	146	146
q7	925	971	733	733
q8	9321	1394	1299	1299
q9	5594	5412	5325	5325
q10	6252	2091	1798	1798
q11	469	264	256	256
q12	623	421	294	294
q13	18164	3311	2698	2698
q14	294	284	261	261
q15	q16	877	868	795	795
q17	964	1093	697	697
q18	6438	5652	5617	5617
q19	1158	1278	998	998
q20	518	413	267	267
q21	4565	2433	1955	1955
q22	483	422	332	332
Total cold run time: 97275 ms
Total hot run time: 29479 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4980	4826	4916	4826
q2	q3	4646	4784	4200	4200
q4	2158	2200	1413	1413
q5	4983	4994	5257	4994
q6	200	177	135	135
q7	2034	1862	1635	1635
q8	3419	3094	3203	3094
q9	8551	8569	8456	8456
q10	4509	4498	4251	4251
q11	616	431	407	407
q12	694	756	513	513
q13	3237	3622	2859	2859
q14	310	308	281	281
q15	q16	806	984	707	707
q17	1359	1309	1272	1272
q18	7963	7081	7158	7081
q19	1147	1135	1131	1131
q20	2238	2208	1946	1946
q21	6174	5486	4914	4914
q22	546	506	425	425
Total cold run time: 60570 ms
Total hot run time: 54540 ms

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29558 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3636d6136379dbf64a9c53a1ce970959e2c6acb2, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17631	3917	3891	3891
q2	q3	10706	855	594	594
q4	4661	465	345	345
q5	7451	1357	1137	1137
q6	193	172	144	144
q7	912	938	758	758
q8	9301	1384	1272	1272
q9	5515	5379	5288	5288
q10	6249	2058	1796	1796
q11	472	263	256	256
q12	629	415	296	296
q13	18135	3297	2701	2701
q14	291	286	267	267
q15	q16	865	871	789	789
q17	912	1064	742	742
q18	6461	5726	5594	5594
q19	1219	1210	1094	1094
q20	502	402	262	262
q21	4955	2396	1977	1977
q22	468	431	355	355
Total cold run time: 97528 ms
Total hot run time: 29558 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4810	4784	4835	4784
q2	q3	4623	4741	4215	4215
q4	2109	2165	1413	1413
q5	4982	4944	5240	4944
q6	195	169	139	139
q7	2021	1830	1627	1627
q8	3322	3118	3098	3098
q9	8388	8481	8430	8430
q10	4509	4491	4295	4295
q11	623	423	380	380
q12	707	754	526	526
q13	3190	3646	2848	2848
q14	303	305	279	279
q15	q16	779	769	687	687
q17	1331	1430	1305	1305
q18	7871	7027	7087	7027
q19	1153	1167	1141	1141
q20	2255	2236	1942	1942
q21	6034	5388	4869	4869
q22	554	502	411	411
Total cold run time: 59759 ms
Total hot run time: 54360 ms

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 170207 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3636d6136379dbf64a9c53a1ce970959e2c6acb2, data reload: false
query5	4311	679	524	524
query6	326	220	202	202
query7	4261	565	305	305
query8	320	230	211	211
query9	8809	4015	3983	3983
query10	442	349	296	296
query11	5780	2467	2242	2242
query12	180	130	134	130
query13	1299	609	448	448
query14	5969	5409	5029	5029
query14_1	4417	4397	4394	4394
query15	225	203	183	183
query16	1028	454	426	426
query17	1123	759	611	611
query18	2561	482	344	344
query19	215	204	162	162
query20	142	130	126	126
query21	209	138	117	117
query22	13633	13583	14873	13583
query23	17372	16528	16157	16157
query23_1	16271	16238	16208	16208
query24	7500	1721	1341	1341
query24_1	1320	1330	1342	1330
query25	563	502	418	418
query26	1317	315	168	168
query27	2712	610	334	334
query28	4447	1957	1950	1950
query29	981	634	515	515
query30	303	232	193	193
query31	1108	1060	935	935
query32	84	69	71	69
query33	520	336	281	281
query34	1168	1180	673	673
query35	769	778	657	657
query36	1337	1306	1214	1214
query37	153	99	89	89
query38	3212	3133	3055	3055
query39	923	925	884	884
query39_1	876	869	875	869
query40	234	155	132	132
query41	62	75	58	58
query42	104	108	104	104
query43	323	325	283	283
query44 query45	208	200	190	190
query46	1093	1168	728	728
query47	2311	2292	2178	2178
query48	393	423	299	299
query49	642	544	419	419
query50	701	283	211	211
query51	4261	4266	4171	4171
query52	104	105	93	93
query53	247	271	200	200
query54	315	261	246	246
query55	97	89	84	84
query56	300	310	307	307
query57	1398	1398	1303	1303
query58	292	284	248	248
query59	1579	1633	1448	1448
query60	331	330	318	318
query61	162	156	160	156
query62	677	613	554	554
query63	238	205	195	195
query64	2434	830	721	721
query65 query66	1718	513	414	414
query67	29437	30007	29927	29927
query68 query69	472	342	315	315
query70	1024	982	999	982
query71	321	277	273	273
query72	3163	3039	2481	2481
query73	845	799	431	431
query74	5053	4942	4723	4723
query75	2755	2663	2317	2317
query76	2296	1145	755	755
query77	411	426	343	343
query78	12925	12914	12330	12330
query79	1492	992	747	747
query80	779	586	477	477
query81	468	282	242	242
query82	1406	160	120	120
query83	328	286	250	250
query84	262	144	112	112
query85	874	495	444	444
query86	428	347	327	327
query87	3445	3369	3223	3223
query88	3615	2678	2671	2671
query89	438	374	336	336
query90	1829	183	180	180
query91	176	168	139	139
query92	75	80	68	68
query93	967	931	578	578
query94	626	362	295	295
query95	669	385	447	385
query96	1099	774	331	331
query97	2673	2681	2544	2544
query98	242	227	225	225
query99	1121	1142	1013	1013
Total cold run time: 253044 ms
Total hot run time: 170207 ms

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 170454 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3636d6136379dbf64a9c53a1ce970959e2c6acb2, data reload: false
query5	4328	654	515	515
query6	336	219	201	201
query7	4218	583	304	304
query8	316	233	218	218
query9	8837	4047	3989	3989
query10	457	335	316	316
query11	5803	2405	2189	2189
query12	180	128	156	128
query13	1245	603	431	431
query14	6003	5336	5038	5038
query14_1	4336	4336	4326	4326
query15	208	204	182	182
query16	1020	475	420	420
query17	1146	763	602	602
query18	2474	484	349	349
query19	211	207	164	164
query20	144	129	132	129
query21	222	144	118	118
query22	13592	13841	14701	13841
query23	17423	16546	16198	16198
query23_1	16187	16287	16257	16257
query24	7496	1760	1363	1363
query24_1	1375	1381	1369	1369
query25	630	520	445	445
query26	1294	321	172	172
query27	2689	593	353	353
query28	4391	2005	1967	1967
query29	1002	647	537	537
query30	303	235	201	201
query31	1108	1070	935	935
query32	84	75	77	75
query33	552	363	302	302
query34	1180	1143	636	636
query35	767	790	692	692
query36	1318	1373	1153	1153
query37	157	106	89	89
query38	3185	3116	3045	3045
query39	922	936	890	890
query39_1	858	856	884	856
query40	238	204	131	131
query41	62	61	58	58
query42	106	105	103	103
query43	317	319	282	282
query44 query45	204	195	193	193
query46	1061	1164	729	729
query47	2258	2236	2094	2094
query48	401	397	300	300
query49	623	521	423	423
query50	704	288	216	216
query51	4282	4227	4236	4227
query52	103	105	94	94
query53	234	278	203	203
query54	311	278	249	249
query55	94	89	85	85
query56	295	294	300	294
query57	1401	1370	1306	1306
query58	287	286	265	265
query59	1542	1584	1400	1400
query60	340	331	312	312
query61	162	157	161	157
query62	671	620	563	563
query63	236	196	205	196
query64	2477	879	684	684
query65 query66	1757	525	386	386
query67	30007	29943	29846	29846
query68 query69	456	333	310	310
query70	1042	975	992	975
query71	310	282	267	267
query72	3056	2703	2437	2437
query73	859	751	437	437
query74	5080	4868	4725	4725
query75	2759	2693	2322	2322
query76	2291	1131	800	800
query77	431	432	355	355
query78	12992	12926	12331	12331
query79	1497	983	748	748
query80	1369	585	496	496
query81	497	272	236	236
query82	1340	161	122	122
query83	366	278	244	244
query84	258	145	111	111
query85	927	532	447	447
query86	456	323	329	323
query87	3397	3362	3186	3186
query88	3562	2699	2677	2677
query89	440	383	331	331
query90	2010	173	181	173
query91	179	167	142	142
query92	74	76	72	72
query93	940	972	570	570
query94	704	338	311	311
query95	655	462	359	359
query96	991	776	338	338
query97	2680	2664	2547	2547
query98	237	242	228	228
query99	1121	1098	962	962
Total cold run time: 254059 ms
Total hot run time: 170454 ms

@CalvinKirs

CalvinKirs commented May 13, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29691 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3636d6136379dbf64a9c53a1ce970959e2c6acb2, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17710	3992	3933	3933
q2	q3	10737	862	600	600
q4	4667	459	348	348
q5	7475	1322	1128	1128
q6	185	166	139	139
q7	898	948	744	744
q8	9299	1369	1277	1277
q9	5543	5396	5347	5347
q10	6245	2079	1803	1803
q11	462	257	254	254
q12	624	412	289	289
q13	18129	3298	2709	2709
q14	288	286	261	261
q15	q16	871	879	793	793
q17	892	1065	808	808
q18	6396	5693	5599	5599
q19	1148	1246	1093	1093
q20	522	396	268	268
q21	4361	2371	1960	1960
q22	506	411	338	338
Total cold run time: 96958 ms
Total hot run time: 29691 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4678	4711	4974	4711
q2	q3	4646	4776	4180	4180
q4	2126	2180	1397	1397
q5	5008	5030	5247	5030
q6	196	173	138	138
q7	2099	1759	1650	1650
q8	3335	3079	3120	3079
q9	8400	8452	8525	8452
q10	4474	4483	4238	4238
q11	595	462	393	393
q12	701	748	514	514
q13	3218	3575	2947	2947
q14	304	305	271	271
q15	q16	767	798	720	720
q17	1496	1295	1229	1229
q18	8041	7070	7090	7070
q19	1162	1134	1162	1134
q20	2245	2239	1940	1940
q21	6140	5425	4870	4870
q22	541	497	424	424
Total cold run time: 60172 ms
Total hot run time: 54387 ms

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 171320 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3636d6136379dbf64a9c53a1ce970959e2c6acb2, data reload: false
query5	4331	643	524	524
query6	336	225	197	197
query7	4228	500	289	289
query8	325	239	220	220
query9	8818	4023	4016	4016
query10	453	333	295	295
query11	5775	2432	2170	2170
query12	191	129	128	128
query13	1355	587	411	411
query14	5948	5393	5102	5102
query14_1	4376	4383	4342	4342
query15	209	207	181	181
query16	1028	451	348	348
query17	1132	763	615	615
query18	2481	482	351	351
query19	210	195	154	154
query20	140	129	130	129
query21	209	135	119	119
query22	13549	13840	14745	13840
query23	17192	16475	16246	16246
query23_1	16463	16366	16213	16213
query24	7458	1755	1350	1350
query24_1	1350	1365	1364	1364
query25	560	472	434	434
query26	1312	319	168	168
query27	2698	604	340	340
query28	4416	1978	1902	1902
query29	987	619	495	495
query30	300	229	194	194
query31	1086	1070	921	921
query32	83	73	72	72
query33	531	348	303	303
query34	1150	1131	640	640
query35	753	772	658	658
query36	1315	1295	1246	1246
query37	145	97	89	89
query38	3168	3133	3022	3022
query39	929	929	899	899
query39_1	878	904	916	904
query40	244	150	139	139
query41	66	61	60	60
query42	105	107	110	107
query43	318	319	277	277
query44 query45	223	202	192	192
query46	1056	1186	767	767
query47	2327	2351	2220	2220
query48	413	418	290	290
query49	636	534	427	427
query50	714	275	223	223
query51	4341	4261	4147	4147
query52	105	107	95	95
query53	250	276	212	212
query54	303	273	251	251
query55	92	89	84	84
query56	301	314	302	302
query57	1409	1370	1306	1306
query58	291	272	259	259
query59	1564	1598	1368	1368
query60	347	339	324	324
query61	162	162	160	160
query62	679	624	563	563
query63	242	197	204	197
query64	2427	843	678	678
query65 query66	1743	524	422	422
query67	29930	30000	29975	29975
query68 query69	473	334	320	320
query70	1058	1022	977	977
query71	310	281	270	270
query72	3253	2930	2691	2691
query73	842	735	436	436
query74	5062	4948	4745	4745
query75	2804	2697	2342	2342
query76	2288	1115	808	808
query77	422	458	355	355
query78	12861	13000	12322	12322
query79	1438	991	732	732
query80	679	587	469	469
query81	458	276	245	245
query82	1310	166	126	126
query83	354	279	250	250
query84	257	144	118	118
query85	866	533	448	448
query86	399	325	344	325
query87	3413	3338	3188	3188
query88	3546	2669	2661	2661
query89	421	388	344	344
query90	2011	179	177	177
query91	181	167	141	141
query92	75	81	71	71
query93	962	938	570	570
query94	524	360	319	319
query95	670	468	366	366
query96	1037	790	353	353
query97	2718	2698	2593	2593
query98	238	232	231	231
query99	1117	1107	994	994
Total cold run time: 252989 ms
Total hot run time: 171320 ms

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 57.14% (8/14) 🎉
Increment coverage report
Complete coverage report

1 similar comment
@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 57.14% (8/14) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

hello-stephen commented May 13, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 42.86% (6/14) 🎉
Increment coverage report
Complete coverage report

@CalvinKirs
[fix](fe) Reject JSONB and variant distribution columns
cb5fd75
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Hash distribution column validation missed JSONB and VARIANT columns, allowing CREATE TABLE statements such as DISTRIBUTED BY HASH(jsonb_col) even though those types are rejected in key and partition paths. This change centralizes hash distribution type validation in HashDistributionInfo and reuses it when building catalog distribution info, including Nereids create-table execution after descriptor translation. Variant regression assertions are updated to expect the unified distribution-column error message.
### Release note
Reject JSONB and VARIANT columns as hash distribution columns during CREATE TABLE analysis.
### Check List (For Author)
- Test: Unit Test, static check, and Regression test attempt
- Unit Test: mvn -pl fe-core -am -Dmaven.build.cache.enabled=false -Dtest=org.apache.doris.nereids.trees.plans.commands.info.DistributionDescriptorTest,org.apache.doris.nereids.trees.plans.CreateTableCommandTest#testAbnormal -DfailIfNoTests=false -Dfe_ut_parallel=0 test
- Unit Test: mvn -pl fe-core -am -Dtest=org.apache.doris.catalog.CreateTableTest#testAbnormal -DfailIfNoTests=false -Dfe_ut_parallel=0 test
- Static check: git diff --check
- Regression test: ./run-regression-test.sh --run -d jsonb_p0 -s test_jsonb_distribution_column (attempted against the existing 127.0.0.1:9030 cluster; it failed because that cluster was not rebuilt from this branch and still accepted HASH(JSONB), confirmed by direct mysql SHOW CREATE TABLE)
- Behavior changed: Yes. CREATE TABLE now rejects JSONB and VARIANT hash distribution columns.
- Does this need documentation: No
@github-actionsgithub-actionsBot removed the approved Indicates a PR has been approved by one committer. label May 14, 2026
@CalvinKirs

CalvinKirs commented May 14, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

@hello-stephen

hello-stephen commented May 14, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29533 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cb5fd7501df567a8d7d742b47e143fb37c627cc3, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17635	3869	4030	3869
q2	q3	10703	881	627	627
q4	4656	473	349	349
q5	7547	1324	1153	1153
q6	254	170	142	142
q7	949	948	761	761
q8	10321	1455	1248	1248
q9	6615	5390	5319	5319
q10	6308	2086	1820	1820
q11	483	271	268	268
q12	688	418	289	289
q13	18140	3230	2731	2731
q14	290	284	259	259
q15	q16	904	864	789	789
q17	981	1102	763	763
q18	6550	5702	5597	5597
q19	1359	1235	1092	1092
q20	522	401	276	276
q21	4591	2305	1879	1879
q22	425	353	302	302
Total cold run time: 99921 ms
Total hot run time: 29533 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4203	4168	4131	4131
q2	q3	4647	4797	4160	4160
q4	2093	2134	1365	1365
q5	4990	4946	5261	4946
q6	186	165	132	132
q7	2024	2073	1691	1691
q8	3467	3192	3158	3158
q9	8600	8425	8415	8415
q10	4513	4511	4243	4243
q11	619	439	386	386
q12	694	751	524	524
q13	3297	3648	2964	2964
q14	295	302	269	269
q15	q16	785	799	691	691
q17	1379	1359	1266	1266
q18	8170	7108	7124	7108
q19	1167	1178	1104	1104
q20	2246	2256	1963	1963
q21	6163	5539	5001	5001
q22	562	521	434	434
Total cold run time: 60100 ms
Total hot run time: 53951 ms

@hello-stephen

hello-stephen commented May 14, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 170629 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb5fd7501df567a8d7d742b47e143fb37c627cc3, data reload: false
query5	4333	665	512	512
query6	346	219	204	204
query7	4299	564	297	297
query8	326	223	216	216
query9	8840	4023	3991	3991
query10	462	352	303	303
query11	5831	2392	2213	2213
query12	210	127	129	127
query13	1285	593	415	415
query14	6555	5345	5064	5064
query14_1	4337	4340	4346	4340
query15	213	198	180	180
query16	1023	432	420	420
query17	1137	734	604	604
query18	2735	482	355	355
query19	212	203	162	162
query20	136	129	127	127
query21	208	146	119	119
query22	13569	13872	14611	13872
query23	17648	16471	16280	16280
query23_1	16298	16236	16271	16236
query24	7949	1840	1428	1428
query24_1	1359	1340	1366	1340
query25	543	470	429	429
query26	1299	336	170	170
query27	2691	594	329	329
query28	4401	1970	1959	1959
query29	967	615	511	511
query30	304	241	196	196
query31	1126	1062	923	923
query32	84	70	68	68
query33	546	356	313	313
query34	1182	1116	628	628
query35	801	795	660	660
query36	1388	1363	1179	1179
query37	155	102	87	87
query38	3204	3122	3070	3070
query39	940	930	899	899
query39_1	874	900	883	883
query40	243	161	142	142
query41	69	67	70	67
query42	120	104	105	104
query43	330	330	296	296
query44 query45	213	206	195	195
query46	1102	1184	728	728
query47	2340	2297	2171	2171
query48	426	410	312	312
query49	648	550	466	466
query50	706	310	220	220
query51	4278	4230	4225	4225
query52	108	110	95	95
query53	262	280	214	214
query54	340	289	273	273
query55	100	101	87	87
query56	312	330	307	307
query57	1417	1404	1333	1333
query58	320	299	275	275
query59	1569	1605	1438	1438
query60	354	342	336	336
query61	181	181	180	180
query62	678	623	553	553
query63	251	208	211	208
query64	2414	903	753	753
query65 query66	1686	502	379	379
query67	30163	29480	29918	29480
query68 query69	461	341	298	298
query70	971	972	955	955
query71	315	277	266	266
query72	2979	2745	2444	2444
query73	831	744	441	441
query74	5069	4901	4721	4721
query75	2803	2678	2316	2316
query76	2266	1129	790	790
query77	408	425	343	343
query78	12863	13018	12487	12487
query79	2973	986	716	716
query80	1806	573	481	481
query81	527	277	244	244
query82	947	165	127	127
query83	321	277	246	246
query84	269	140	105	105
query85	894	509	436	436
query86	461	358	317	317
query87	3418	3347	3216	3216
query88	3537	2668	2657	2657
query89	442	383	336	336
query90	1887	185	177	177
query91	178	165	134	134
query92	81	76	70	70
query93	1249	967	568	568
query94	701	333	306	306
query95	657	375	434	375
query96	996	782	334	334
query97	2714	2762	2540	2540
query98	240	229	229	229
query99	1117	1119	989	989
Total cold run time: 257595 ms
Total hot run time: 170629 ms

@hello-stephen

hello-stephen commented May 14, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29639 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cb5fd7501df567a8d7d742b47e143fb37c627cc3, data reload: false
------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	17791	3876	3831	3831
q2	q3	10718	880	615	615
q4	4656	456	352	352
q5	7451	1331	1149	1149
q6	216	170	136	136
q7	938	937	764	764
q8	9616	1411	1290	1290
q9	6554	5479	5372	5372
q10	6319	2087	1833	1833
q11	471	259	255	255
q12	690	414	289	289
q13	18155	3346	2776	2776
q14	285	283	258	258
q15	q16	896	875	791	791
q17	1022	979	743	743
q18	6413	5619	5627	5619
q19	1189	1206	1128	1128
q20	501	405	261	261
q21	4820	2247	1873	1873
q22	417	350	304	304
Total cold run time: 99118 ms
Total hot run time: 29639 ms
----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL ============================================
q1	4159	4089	4096	4089
q2	q3	4639	4790	4188	4188
q4	2101	2242	1376	1376
q5	4920	4960	5218	4960
q6	186	163	130	130
q7	2034	1974	1726	1726
q8	3453	3193	3216	3193
q9	8555	8468	8481	8468
q10	4479	4511	4261	4261
q11	626	428	396	396
q12	715	743	519	519
q13	3192	3613	2946	2946
q14	304	307	278	278
q15	q16	765	802	712	712
q17	1335	1349	1270	1270
q18	8155	7091	7142	7091
q19	1154	1158	1157	1157
q20	2224	2245	2025	2025
q21	6125	5429	4867	4867
q22	572	534	458	458
Total cold run time: 59693 ms
Total hot run time: 54110 ms

@hello-stephen

hello-stephen commented May 14, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 172048 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cb5fd7501df567a8d7d742b47e143fb37c627cc3, data reload: false
query5	4330	644	499	499
query6	335	241	208	208
query7	4263	580	297	297
query8	338	229	227	227
query9	8844	4135	4044	4044
query10	496	344	307	307
query11	5927	2433	2244	2244
query12	190	132	133	132
query13	1286	577	400	400
query14	6835	5417	5077	5077
query14_1	4420	4389	4370	4370
query15	221	210	184	184
query16	987	478	450	450
query17	1351	793	637	637
query18	2752	502	377	377
query19	262	206	170	170
query20	142	137	132	132
query21	221	145	120	120
query22	13646	14069	14594	14069
query23	17434	16653	16280	16280
query23_1	16343	16246	16417	16246
query24	8027	1737	1343	1343
query24_1	1333	1359	1342	1342
query25	549	497	430	430
query26	1310	309	169	169
query27	2673	590	337	337
query28	4336	1930	1918	1918
query29	996	646	526	526
query30	300	232	196	196
query31	1135	1073	942	942
query32	94	72	70	70
query33	540	337	288	288
query34	1161	1164	648	648
query35	780	781	656	656
query36	1322	1318	1168	1168
query37	148	96	86	86
query38	3176	3145	3056	3056
query39	923	924	878	878
query39_1	879	859	885	859
query40	231	155	137	137
query41	64	63	59	59
query42	107	103	109	103
query43	330	324	286	286
query44 query45	208	206	188	188
query46	1023	1171	736	736
query47	2304	2301	2147	2147
query48	388	409	285	285
query49	639	514	432	432
query50	697	283	252	252
query51	4289	4300	4251	4251
query52	109	104	97	97
query53	256	274	201	201
query54	317	283	257	257
query55	91	88	82	82
query56	298	303	302	302
query57	1437	1390	1302	1302
query58	292	270	264	264
query59	1513	1626	1440	1440
query60	337	334	328	328
query61	156	153	148	148
query62	687	621	563	563
query63	248	201	205	201
query64	2320	815	670	670
query65 query66	1680	495	393	393
query67	29976	29980	29833	29833
query68 query69	466	333	344	333
query70	1019	982	939	939
query71	303	286	271	271
query72	2914	2735	2641	2641
query73	863	762	418	418
query74	5100	4911	4739	4739
query75	2752	2648	2312	2312
query76	2318	1142	770	770
query77	405	443	351	351
query78	12949	12928	12244	12244
query79	1530	956	745	745
query80	690	571	486	486
query81	453	278	243	243
query82	1348	162	128	128
query83	358	269	246	246
query84	266	140	114	114
query85	836	546	465	465
query86	410	311	330	311
query87	3406	3349	3219	3219
query88	3556	2672	2649	2649
query89	422	378	338	338
query90	1938	185	187	185
query91	183	167	135	135
query92	78	75	76	75
query93	961	960	566	566
query94	531	358	301	301
query95	681	471	346	346
query96	1069	746	342	342
query97	2743	2683	2615	2615
query98	245	229	230	229
query99	1123	1103	985	985
Total cold run time: 255128 ms
Total hot run time: 172048 ms

@morrySnow

morrySnow commented May 14, 2026

Copy link
Copy Markdown
Contributor

/review

github-actions[bot]
github-actionsBot reviewed May 14, 2026
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking issues found in the changed files. The new centralized validation in HashDistributionInfo is used by HashDistributionDesc.toDistributionInfo(), so both catalog and Nereids create-table flows reject JSONB/VARIANT hash distribution columns before table creation. The regression and unit coverage exercise JSONB and VARIANT rejection paths, and the existing variant regression expectations are updated consistently.

Critical checkpoint conclusions:

  • Goal/test coverage: The PR addresses the stated missing validation and adds targeted unit/regression tests. The regression test was reported as attempted but not successfully run on a rebuilt branch cluster.
  • Scope/focus: The implementation is small and focused; it reuses the existing distribution conversion choke point instead of adding parallel checks.
  • Concurrency/lifecycle/config/compatibility: No new concurrency, lifecycle, config, persistence, or wire/storage compatibility concerns identified.
  • Parallel paths: Legacy catalog conversion and Nereids descriptor translation both reach HashDistributionDesc.toDistributionInfo(); no separate unchecked create-table path was found in the PR scope.
  • Data correctness/error handling: Invalid distribution types fail with DdlException before metadata construction, avoiding persisted invalid table metadata.
  • Observability/performance: No additional observability needed; no hot-path performance issue identified.
  • User focus: No additional user-provided review focus was specified.

@hello-stephen

hello-stephen commented May 14, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 20.69% (6/29) 🎉
Increment coverage report
Complete coverage report

@starocean999
starocean999 merged commit d80ead8 into apache:masterMay 15, 2026
32 checks passed
@CalvinKirsCalvinKirs mentioned this pull request May 27, 2026
Merged
yiguolei pushed a commit that referenced this pull request May 27, 2026
@CalvinKirs
branch-4.1: [fix](fe) Reject JSONB and variant distribution columns (#…
9449c37
…63211) (#63725)
#63211
@CalvinKirsCalvinKirs mentioned this pull request May 28, 2026
Merged
morningman pushed a commit that referenced this pull request May 29, 2026
@CalvinKirs
branch-4.0: [fix](fe) Reject JSONB and variant distribution columns (#…
80adba3
…63211) (#63847)
Related PR: #63211
@morningmanmorningman mentioned this pull request Jun 3, 2026
Closed
zhaorongsheng pushed a commit to zhaorongsheng/doris that referenced this pull request Jun 4, 2026
@CalvinKirszhaorongsheng
[fix](fe) Reject JSONB and variant distribution columns (apache#63211)
4d6e2bb
Problem Summary: Hash distribution column validation missed JSONB and
VARIANT columns, allowing CREATE TABLE statements such as DISTRIBUTED BY
HASH(jsonb_col) even though those types are rejected in key and
partition paths. This change centralizes hash distribution type
validation and reuses it from both catalog and Nereids create-table
paths.
### Release note
Reject JSONB and VARIANT columns as hash distribution columns during
CREATE TABLE analysis.
@yiguoleiyiguolei mentioned this pull request Jun 14, 2026
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@starocean999starocean999starocean999 approved these changes

Assignees

No one assigned

Labels

dev/3.1.xdev/4.0.6-mergeddev/4.1.2-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@CalvinKirs@hello-stephen@morrySnow@starocean999@morningman@yiguolei