[fix](partial update) Fix session variable `enable_unique_key_partial_update` will affect the behavior of insert statement when the target table is not unique table by bobhan1 · Pull Request #28321 · apache/doris

bobhan1 · 2023-12-13T03:36:34Z

Proposed changes

the session variable enable_unique_key_partial_update should be ignored if the target table is not unique table.
check the partial update constraints in the following order:

if the target table is a merge-on-read table, report "Partial update is only allowed on unique table with merge-on-write enabled."
if the users don't specify the columns, report "You must explicitly specify the columns to be updated when updating partial columns using the INSERT statement."
if there exist key column that doesn't be contained in the specified columns, report "Partial update should include all key columns, missing: xxx"

fix some bugs.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

bobhan1 · 2023-12-13T03:37:05Z

run buildall

doris-robot · 2023-12-13T05:34:53Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit ac0ab3bd28ba7d1a0f98382130386841deac8b55, data reload: false
run tpch-sf100 query with default conf and session variables
q1	51482	31823	31833	31823
q2	387	134	135	134
q3	1581	1262	1225	1225
q4	1119	932	892	892
q5	3201	3218	3266	3218
q6	257	145	144	144
q7	1001	499	482	482
q8	2181	2204	2194	2194
q9	6947	6884	6912	6884
q10	3256	3318	3284	3284
q11	327	210	197	197
q12	349	208	202	202
q13	4548	3803	3808	3803
q14	379	315	320	315
q15	823	1303	1327	1303
q16	437	379	390	379
q17	1038	554	546	546
q18	7347	6891	7462	6891
q19	1532	1347	1554	1347
q20	607	294	326	294
q21	3108	2625	2666	2625
q22	349	286	295	286
Total cold run time: 92256 ms
Total hot run time: 68468 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	30801	31847	31822	31822
q2	276	163	172	163
q3	3628	3627	3625	3625
q4	2394	2378	2382	2378
q5	5775	5794	5832	5794
q6	240	136	131	131
q7	3046	3414	3365	3365
q8	3549	3548	3546	3546
q9	9219	9193	9140	9140
q10	3978	4058	4048	4048
q11	798	945	1351	945
q12	760	610	592	592
q13	4285	3580	3541	3541
q14	497	428	432	428
q15	1140	1326	1330	1326
q16	516	464	488	464
q17	1886	1849	1853	1849
q18	8773	8529	8987	8529
q19	1744	1801	1774	1774
q20	2889	2601	2648	2601
q21	6570	6147	6147	6147
q22	513	436	410	410
Total cold run time: 93277 ms
Total hot run time: 92618 ms

bobhan1 · 2023-12-13T06:11:12Z

run clickbench

doris-robot · 2023-12-13T06:59:58Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.24 seconds
stream load tsv: 577 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17219981480 Bytes

doris-robot · 2023-12-13T07:01:50Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit ac0ab3bd28ba7d1a0f98382130386841deac8b55, data reload: false
run tpch-sf100 query with default conf and session variables
q1	49973	31822	31795	31795
q2	386	138	137	137
q3	1583	1231	1220	1220
q4	1109	905	947	905
q5	3203	3198	3193	3193
q6	255	143	140	140
q7	993	495	480	480
q8	2175	2220	2210	2210
q9	6895	6936	6906	6906
q10	3245	3302	3315	3302
q11	340	195	200	195
q12	354	199	201	199
q13	4564	3755	3821	3755
q14	428	318	315	315
q15	775	1320	1332	1320
q16	434	388	382	382
q17	1033	562	559	559
q18	7446	6940	7966	6940
q19	1541	1412	1422	1412
q20	652	451	366	366
q21	3097	2647	2732	2647
q22	351	282	286	282
Total cold run time: 90832 ms
Total hot run time: 68660 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	30801	32022	31643	31643
q2	280	165	173	165
q3	3688	3601	3622	3601
q4	2407	2390	2390	2390
q5	5794	5791	5806	5791
q6	242	134	135	134
q7	3051	3401	3369	3369
q8	3544	3555	3550	3550
q9	9176	9149	9170	9149
q10	3981	4054	4037	4037
q11	804	927	1348	927
q12	766	623	595	595
q13	4312	3567	3520	3520
q14	509	446	430	430
q15	1126	1339	1333	1333
q16	509	441	452	441
q17	1911	1851	1847	1847
q18	8845	8362	9338	8362
q19	1757	1776	1766	1766
q20	2896	2564	2586	2564
q21	6569	6194	6167	6167
q22	510	421	421	421
Total cold run time: 93478 ms
Total hot run time: 92202 ms

zhannngchen

LGTM

github-actions · 2023-12-13T08:36:05Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-12-13T08:36:07Z

PR approved by anyone and no changes requested.

dataroaring

LGTM

…_update` will affect the behavior of insert statement when the target table is not unique table (apache#28321)

…28321 #28358 (#28844)

…_update` will affect the behavior of insert statement when the target table is not unique table (apache#28321)

bobhan1 added 3 commits December 13, 2023 11:35

fix BindSink
22fb125

tmp
cbafc4b

update
ac0ab3b

zhannngchen approved these changes Dec 13, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Dec 13, 2023

github-actionsBot added the reviewed label Dec 13, 2023

zhannngchen requested a review from dataroaring December 13, 2023 09:00

dataroaring added the dev/2.0.4 label Dec 13, 2023

dataroaring approved these changes Dec 13, 2023
View reviewed changes

dataroaring merged commit ac262fa into apache:masterDec 13, 2023

bobhan1 mentioned this pull request Dec 13, 2023
[fix](partial update) Fix some bugs about partial update #28358
Merged

seawinde pushed a commit to seawinde/doris that referenced this pull request Dec 14, 2023
[fix](partial update) Fix session variable `enable_unique_key_partial…
8966aa5
…_update` will affect the behavior of insert statement when the target table is not unique table (apache#28321)

wm1581066 added the usercase Important user case type label label Dec 16, 2023

bobhan1 mentioned this pull request Dec 22, 2023
[branch-2.0-pick](partial update) Pick some PRs about partial update #28844
Merged

xiaokang pushed a commit that referenced this pull request Dec 23, 2023
[branch-2.0-pick](partial update) Pick some PRs about partial update#…
a15b289
…28321 #28358 (#28844)

xiaokang added dev/2.0.4-merged and removed dev/2.0.4 labels Dec 23, 2023

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
[fix](partial update) Fix session variable `enable_unique_key_partial…
e2b05c1
…_update` will affect the behavior of insert statement when the target table is not unique table (apache#28321)

This was referenced Apr 15, 2024
[Bug] can not insert into values when set global enable_unique_key_partial_update = true #33649
Closed
[fix](partial update) only unique table with MOW insert with target columns can consider be a partial update #33656
Merged

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
[fix](partial update) Fix session variable `enable_unique_key_partial…
9934d51
…_update` will affect the behavior of insert statement when the target table is not unique table (apache#28321)

Navigation Menu

Uh oh!

[fix](partial update) Fix session variable `enable_unique_key_partial_update` will affect the behavior of insert statement when the target table is not unique table - #28321

[fix](partial update) Fix session variable `enable_unique_key_partial_update` will affect the behavior of insert statement when the target table is not unique table#28321
dataroaring merged 3 commits into
apache:masterfrom
bobhan1:fix-not-specify-columns

bobhan1 commented Dec 13, 2023•
edited
Loading

Uh oh!

bobhan1 commented Dec 13, 2023

Uh oh!

doris-robot commented Dec 13, 2023

Uh oh!

bobhan1 commented Dec 13, 2023

Uh oh!

doris-robot commented Dec 13, 2023

Uh oh!

doris-robot commented Dec 13, 2023

Uh oh!

zhannngchen left a comment

Uh oh!

github-actionsBot commented Dec 13, 2023

Uh oh!

github-actionsBot commented Dec 13, 2023

Uh oh!

dataroaring left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

Conversation

bobhan1 commented Dec 13, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

bobhan1 commented Dec 13, 2023

Uh oh!

doris-robot commented Dec 13, 2023

Uh oh!

bobhan1 commented Dec 13, 2023

Uh oh!

doris-robot commented Dec 13, 2023

Uh oh!

doris-robot commented Dec 13, 2023

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Dec 13, 2023

Uh oh!

github-actionsBot commented Dec 13, 2023

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bobhan1 commented Dec 13, 2023•
edited
Loading