[Fix](Txn) Fix transaction write to sequence column error by Yukang-Lian · Pull Request #26748 · apache/doris

Yukang-Lian · 2023-11-10T07:00:15Z

Proposed changes

Issue Number: close #xxx

When begin transaction, then insert into value to DORIS_SEQUENCE_COL and DORIS_DELETE_SIGN, there will be an error. This PR fix it.

begin;
insert into test (user_id, group_id, __DORIS_SEQUENCE_COL__) values (1, 5, 5);
ERROR 1105 (HY000): errCode =2, detailMessage = Table test has sequence column, need to specify the sequence column

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

Yukang-Lian · 2023-11-10T07:20:50Z

run buildall

Hastyshell

LGTM

github-actions · 2023-11-10T07:40:03Z

PR approved by anyone and no changes requested.

doris-robot · 2023-11-10T07:53:30Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit af51d2069e3c5d971425bfa32631d222b0837f8b, data reload: false
run tpch-sf100 query with default conf and session variables
q1	5349	5178	5133	5133
q2	370	198	203	198
q3	2108	2050	2042	2042
q4	1484	1430	1435	1430
q5	4158	4155	4133	4133
q6	252	134	133	133
q7	2099	1608	1623	1608
q8	2781	2765	2782	2765
q9	10373	10391	10265	10265
q10	3484	3586	3556	3556
q11	377	251	245	245
q12	470	295	291	291
q13	4576	4097	4173	4097
q14	323	294	289	289
q15	648	583	578	578
q16	695	612	600	600
q17	1160	1106	1078	1078
q18	7885	7373	7481	7373
q19	1697	1703	1714	1703
q20	594	378	361	361
q21	4956	4595	4576	4576
q22	542	456	442	442
Total cold run time: 56381 ms
Total hot run time: 52896 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4991	5158	4975	4975
q2	338	248	258	248
q3	4062	3898	3898	3898
q4	2778	2773	2759	2759
q5	6567	6618	6504	6504
q6	246	127	127	127
q7	3158	2730	2706	2706
q8	4877	4819	4762	4762
q9	17757	17854	17622	17622
q10	4085	4149	4186	4149
q11	784	642	643	642
q12	986	813	807	807
q13	4320	3895	3891	3891
q14	390	363	349	349
q15	629	559	568	559
q16	765	693	682	682
q17	3942	3929	3976	3929
q18	9298	9326	9322	9322
q19	1894	1773	1788	1773
q20	2391	2076	2035	2035
q21	8881	8769	8915	8769
q22	927	826	853	826
Total cold run time: 84066 ms
Total hot run time: 81334 ms

doris-robot · 2023-11-10T08:03:45Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.44 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162380936 Bytes

Yukang-Lian · 2023-11-10T08:15:03Z

run buildall

doris-robot · 2023-11-10T08:50:56Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit d2cb1ffe7b1ac2bb607c11a4c15692afd25ce3dd, data reload: false
run tpch-sf100 query with default conf and session variables
q1	5269	5121	5104	5104
q2	371	190	176	176
q3	2090	2064	2070	2064
q4	1475	1438	1447	1438
q5	4187	4187	4161	4161
q6	255	134	134	134
q7	2102	1601	1610	1601
q8	2802	2769	2768	2768
q9	10452	10405	10406	10405
q10	3525	3555	3561	3555
q11	377	259	246	246
q12	454	301	309	301
q13	4556	4122	4100	4100
q14	327	291	288	288
q15	653	597	562	562
q16	710	611	601	601
q17	1154	1114	1107	1107
q18	7891	7414	7326	7326
q19	1699	1735	1715	1715
q20	593	398	341	341
q21	4945	4592	4575	4575
q22	546	435	435	435
Total cold run time: 56433 ms
Total hot run time: 53003 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5095	5017	5120	5017
q2	385	264	241	241
q3	4045	4020	4022	4020
q4	2798	2763	2753	2753
q5	6527	6415	6499	6415
q6	246	129	138	129
q7	3162	2683	2715	2683
q8	4809	4764	4883	4764
q9	18042	17937	17953	17937
q10	4096	4159	4150	4150
q11	765	645	653	645
q12	1013	840	834	834
q13	4361	3885	3898	3885
q14	390	349	339	339
q15	603	552	583	552
q16	816	695	690	690
q17	3880	4012	3922	3922
q18	9474	9230	9427	9230
q19	1887	1799	1817	1799
q20	2430	2077	2039	2039
q21	8925	8790	8847	8790
q22	947	863	850	850
Total cold run time: 84696 ms
Total hot run time: 81684 ms

doris-robot · 2023-11-10T08:51:54Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.06 seconds
stream load tsv: 556 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162522914 Bytes

zhannngchen

LGTM

zhannngchen · 2023-11-11T01:25:47Z

run feut

github-actions · 2023-11-11T01:26:02Z

PR approved by at least one committer and no changes requested.

Yukang-Lian · 2023-11-11T16:32:41Z

run buildall

Yukang-Lian · 2023-11-13T13:14:32Z

run buildall

doris-robot · 2023-11-13T15:07:52Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 689a987ad9de2503c558e66afebc608145be33a4, data reload: false
run tpch-sf100 query with default conf and session variables
q1	5228	5111	5094	5094
q2	376	147	159	147
q3	2058	2019	2036	2019
q4	1411	1339	1347	1339
q5	3968	3967	3999	3967
q6	257	133	133	133
q7	1425	876	903	876
q8	2776	2785	2762	2762
q9	9580	9521	9420	9420
q10	3434	3538	3521	3521
q11	381	253	261	253
q12	429	287	290	287
q13	4588	4197	4122	4122
q14	324	300	293	293
q15	664	568	551	551
q16	685	586	600	586
q17	1128	1087	1069	1069
q18	8060	7598	7541	7541
q19	1685	1697	1674	1674
q20	541	297	288	288
q21	4680	4329	4350	4329
q22	518	415	417	415
Total cold run time: 54196 ms
Total hot run time: 50686 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5045	5015	5001	5001
q2	383	237	247	237
q3	4084	4007	4062	4007
q4	2792	2789	2768	2768
q5	9545	9511	9558	9511
q6	249	125	125	125
q7	2673	2429	2315	2315
q8	4840	4947	4819	4819
q9	12972	12878	12960	12878
q10	4051	4168	4172	4168
q11	749	645	634	634
q12	983	829	839	829
q13	4298	3872	3831	3831
q14	386	346	350	346
q15	609	569	587	569
q16	787	704	683	683
q17	3865	3931	3988	3931
q18	9750	9340	9401	9340
q19	1886	1777	1788	1777
q20	2379	2058	2055	2055
q21	8946	8862	8732	8732
q22	943	864	890	864
Total cold run time: 82215 ms
Total hot run time: 79420 ms

doris-robot · 2023-11-13T15:18:25Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.82 seconds
stream load tsv: 556 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162064622 Bytes

mymeiyi

LGTM

dataroaring

LGTM

github-actions · 2023-11-14T02:29:43Z

PR approved by at least one committer and no changes requested.

…#36248) Pick #26748 Pick #27780 Pick #33336 --------- Co-authored-by: zhannngchen <48427519+zhannngchen@users.noreply.github.com> Co-authored-by: meiyi <myimeiyi@gmail.com>

…7780 apache#33336" (apache#36248) Pick apache#26748 Pick apache#27780 Pick apache#33336 --------- Co-authored-by: zhannngchen <48427519+zhannngchen@users.noreply.github.com> Co-authored-by: meiyi <myimeiyi@gmail.com>

Yukang-Lian requested a review from Hastyshell November 10, 2023 07:24

Hastyshell approved these changes Nov 10, 2023
View reviewed changes

github-actionsBot added the reviewed label Nov 10, 2023

Yukang-Lian force-pushed the Fix_Txn_Write_To_Sequence_Col branch from 9ecd649 to d2cb1ffCompare November 10, 2023 08:14

zhannngchen previously approved these changes Nov 11, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 11, 2023

Yukang-Lian dismissed zhannngchen’s stale review via 3372e10November 11, 2023 16:29

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Nov 11, 2023

Yukang-Lian force-pushed the Fix_Txn_Write_To_Sequence_Col branch from 3372e10 to afa0ab1Compare November 11, 2023 16:32

Yukang-Lian added 4 commits November 13, 2023 20:58

1
eaacd38

2
5b8d282

3
d6651fa

4
689a987

Yukang-Lian force-pushed the Fix_Txn_Write_To_Sequence_Col branch from afa0ab1 to 689a987Compare November 13, 2023 13:14

mymeiyi approved these changes Nov 14, 2023
View reviewed changes

dataroaring approved these changes Nov 14, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Nov 14, 2023

dataroaring merged commit fef627c into apache:masterNov 14, 2023

seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 14, 2023
[Fix](Txn) Fix transaction write to sequence column error (apache#26748)
8566f78

seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 15, 2023
[Fix](Txn) Fix transaction write to sequence column error (apache#26748)
25ef1ca

XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
[Fix](Txn) Fix transaction write to sequence column error (apache#26748)
f9f3e67

Yukang-Lian mentioned this pull request Jun 15, 2024
[Cherry-Pick](Branch-2.0) Pick some txn bug fix #26748 #27780 #33336" #36248
Merged

Yukang-Lian added a commit to Yukang-Lian/doris that referenced this pull request Jun 15, 2024
[Fix](Txn) Fix transaction write to sequence column error (apache#26748)
31be4f2

Yukang-Lian added a commit to Yukang-Lian/doris that referenced this pull request Jun 15, 2024
[Fix](Txn) Fix transaction write to sequence column error (apache#26748)
9700947

Uh oh!

Conversation

Yukang-Lian commented Nov 10, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

Yukang-Lian commented Nov 10, 2023

Uh oh!

Hastyshell left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Nov 10, 2023

Uh oh!

doris-robot commented Nov 10, 2023

Uh oh!

doris-robot commented Nov 10, 2023

Uh oh!

Yukang-Lian commented Nov 10, 2023

Uh oh!

doris-robot commented Nov 10, 2023

Uh oh!

doris-robot commented Nov 10, 2023

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

zhannngchen commented Nov 11, 2023

Uh oh!

github-actionsBot commented Nov 11, 2023

Uh oh!

Yukang-Lian commented Nov 11, 2023

Uh oh!

Yukang-Lian commented Nov 13, 2023

Uh oh!

doris-robot commented Nov 13, 2023

Uh oh!

doris-robot commented Nov 13, 2023

Uh oh!

mymeiyi left a comment

Choose a reason for hiding this comment

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Nov 14, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Yukang-Lian commented Nov 10, 2023•
edited
Loading