[Bug](schame-change) fix wrong result after reorder mor table by BiteTheDDDDt · Pull Request #32839 · apache/doris

BiteTheDDDDt · 2024-03-26T09:46:44Z

Proposed changes

pick #29045 and part of #28135

*** Query id: 0-0 ***
*** tablet id: 0 ***
*** Aborted at 1710782450 (unix time) try"date -d @1710782450"if you are usingGNU date ***
*** Current BE git commitID: 22810a5d48 ***
*** SIGABRT unknown detail explain (@0x1bf4) received by PID 7156 (TID8591OR0x7f2c9163b700) from PID 7156; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/common/signal_handler.h:4351# 0x00007F33BA645090 in /lib/x86_64-linux-gnu/libc.so.62# raise at ../sysdeps/unix/sysv/linux/raise.c:513# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:814# 0x0000559872835FA9 in /mnt/disk1/selectdb-cloud-dev-asan/cluster0/be/lib/doris_be
5# 0x000055987282B50D in /mnt/disk1/selectdb-cloud-dev-asan/cluster0/be/lib/doris_be
6# google::LogMessage::SendToLog() in /mnt/disk1/selectdb-cloud-dev-asan/cluster0/be/lib/doris_be
7# google::LogMessage::Flush() in /mnt/disk1/selectdb-cloud-dev-asan/cluster0/be/lib/doris_be
8# google::LogMessageFatal::~LogMessageFatal() in /mnt/disk1/selectdb-cloud-dev-asan/cluster0/be/lib/doris_be
9# doris::BetaRowsetWriter::_flush_segment_writer(std::unique_ptr >*, long*) in /mnt/disk1/selectdb-cloud-dev-asan/cluster0/be/lib/doris_be
10# doris::BetaRowsetWriter::flush() at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/olap/rowset/beta_rowset_writer.cpp:48211# doris::VSchemaChangeDirectly::_inner_process(std::shared_ptr, doris::RowsetWriter*, std::shared_ptr, std::shared_ptr) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/olap/schema_change.cpp:50712# doris::SchemaChange::process(std::shared_ptr, doris::RowsetWriter*, std::shared_ptr, std::shared_ptr, std::shared_ptr) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/olap/schema_change.h:14113# doris::cloud::CloudSchemaChange::_convert_historical_rowsets(doris::SchemaChangeParams const&) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/cloud/cloud_schema_change.cpp:25414# doris::cloud::CloudSchemaChange::process_alter_tablet(doris::TAlterTabletReqV2 const&) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/cloud/cloud_schema_change.cpp:15615# doris::EngineAlterTabletTask::execute() at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/olap/task/engine_alter_tablet_task.cpp:5716# doris::StorageEngine::execute_task(doris::EngineTask*) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/cloud/olap/storage_engine.cpp:99817# doris::AlterTableTaskPool::_alter_tablet(doris::TAgentTaskRequest const&, long, doris::TTaskType::type, doris::TFinishTaskRequest*) at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/agent/task_worker_pool.cpp:183218# doris::AlterTableTaskPool::_alter_tablet_worker_thread_callback() at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/agent/task_worker_pool.cpp:178819# doris::AlterTableTaskPool::AlterTableTaskPool(doris::ExecEnv*, doris::TaskWorkerPool::ThreadModel)::$_0::operator()() const at /mnt/disk3/pipeline/repo/selectdb-core_selectdb-cloud-dev/selectdb-core/be/src/agent/task_worker_pool.cpp:1755

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

doris-robot · 2024-03-26T09:46:48Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

BiteTheDDDDt · 2024-03-26T09:48:04Z

run buildall

github-actions

clang-tidy made some suggestions

github-actions · 2024-03-26T09:53:23Z

+ ColumnMapping() = default;
 virtual ~ColumnMapping() = default;

+ bool has_reference() const { return expr != nullptr || ref_column >= 0; }


warning: function 'has_reference' should be marked [[nodiscard]] [modernize-use-nodiscard]
Suggested change
boolhas_reference() const { returnexpr!=nullptr||ref_column >= 0; }
[[nodiscard]] boolhas_reference() const { returnexpr!=nullptr||ref_column >= 0; }

doris-robot · 2024-03-26T10:15:02Z

TeamCity be ut coverage result:
Function Coverage: 37.80% (8046/21286)
Line Coverage: 29.47% (65713/222993)
Region Coverage: 28.93% (33813/116886)
Branch Coverage: 24.78% (17359/70048)
Coverage Report: http://coverage.selectdb-in.cc/coverage/80b76bf41bbe72fdba24346ec3193607eea74519_80b76bf41bbe72fdba24346ec3193607eea74519/report/index.html

doris-robot · 2024-03-26T10:15:25Z

TPC-H: Total hot run time: 50318 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 80b76bf41bbe72fdba24346ec3193607eea74519, data reload: false
------ Round 1 ----------------------------------
q1	17607	4410	4398	4398
q2	2048	153	145	145
q3	10416	1902	1945	1902
q4	10184	1253	1349	1253
q5	8463	4022	4008	4008
q6	234	122	129	122
q7	2059	1588	1600	1588
q8	9253	2760	2751	2751
q9	10985	10663	10670	10663
q10	8693	3529	3493	3493
q11	431	241	246	241
q12	472	297	306	297
q13	18359	3988	4034	3988
q14	359	331	324	324
q15	506	448	463	448
q16	705	602	589	589
q17	1139	1000	985	985
q18	7424	7025	6797	6797
q19	1701	1558	1562	1558
q20	501	304	315	304
q21	4526	4081	4132	4081
q22	500	394	383	383
Total cold run time: 116565 ms
Total hot run time: 50318 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4354	4296	4279	4279
q2	330	227	224	224
q3	4200	4136	4146	4136
q4	2746	2737	2745	2737
q5	7309	7246	7214	7214
q6	232	121	121	121
q7	3247	2853	2833	2833
q8	4359	4523	4556	4523
q9	17150	16880	17034	16880
q10	4265	4257	4302	4257
q11	729	708	686	686
q12	1037	833	853	833
q13	6884	3750	3772	3750
q14	449	418	427	418
q15	503	448	461	448
q16	779	699	702	699
q17	3848	3871	3835	3835
q18	8886	8895	8859	8859
q19	1729	1731	1660	1660
q20	2394	2189	2112	2112
q21	8582	8485	8521	8485
q22	991	983	950	950
Total cold run time: 85003 ms
Total hot run time: 79939 ms

doris-robot · 2024-03-26T10:28:01Z

TPC-DS: Total hot run time: 201784 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 80b76bf41bbe72fdba24346ec3193607eea74519, data reload: false
query1	911	403	384	384
query2	6520	2224	2087	2087
query3	6924	199	202	199
query4	20675	17845	17988	17845
query5	19713	6570	6505	6505
query6	283	207	217	207
query7	4153	296	291	291
query8	251	270	228	228
query9	3198	2738	2652	2652
query10	409	294	318	294
query11	11346	10637	10733	10637
query12	121	81	70	70
query13	5586	653	636	636
query14	17688	13347	13652	13347
query15	359	239	227	227
query16	6460	266	261	261
query17	1725	1465	862	862
query18	2335	416	400	400
query19	214	142	148	142
query20	79	75	77	75
query21	188	93	95	93
query22	5207	5099	4981	4981
query23	32532	32051	31918	31918
query24	7415	6476	6457	6457
query25	529	434	408	408
query26	658	159	161	159
query27	2127	300	296	296
query28	6070	2231	2213	2213
query29	2891	2773	2582	2582
query30	241	165	161	161
query31	922	706	725	706
query32	69	56	60	56
query33	402	252	240	240
query34	853	463	494	463
query35	1111	903	937	903
query36	1528	1460	1280	1280
query37	87	65	64	64
query38	3116	2934	2908	2908
query39	1358	1326	1325	1325
query40	304	100	89	89
query41	35	33	33	33
query42	79	87	79	79
query43	638	605	563	563
query44	1182	717	722	717
query45	237	221	227	221
query46	1268	967	964	964
query47	1805	1802	1709	1709
query48	1005	674	670	670
query49	658	362	367	362
query50	856	615	612	612
query51	4875	4728	4725	4725
query52	89	83	76	76
query53	446	330	313	313
query54	2656	2457	2477	2457
query55	80	82	80	80
query56	211	215	191	191
query57	1223	1091	1057	1057
query58	213	209	189	189
query59	3536	3367	3295	3295
query60	196	209	196	196
query61	85	80	86	80
query62	762	490	506	490
query63	465	332	336	332
query64	2702	1497	1390	1390
query65	3608	3563	3545	3545
query66	777	369	376	369
query67	16255	15647	15987	15647
query68	8327	631	676	631
query69	564	350	356	350
query70	1671	1444	1416	1416
query71	397	299	308	299
query72	6377	3418	3398	3398
query73	746	319	310	310
query74	6325	5909	5824	5824
query75	4680	3714	3690	3690
query76	4725	1184	1164	1164
query77	639	253	250	250
query78	12410	11667	12159	11667
query79	10070	641	660	641
query80	1231	393	381	381
query81	486	237	230	230
query82	1549	98	95	95
query83	163	129	134	129
query84	253	67	69	67
query85	896	286	278	278
query86	334	308	291	291
query87	3228	3067	3081	3067
query88	5031	2308	2310	2308
query89	394	283	307	283
query90	1890	198	218	198
query91	150	114	120	114
query92	62	51	51	51
query93	5850	575	582	575
query94	756	201	208	201
query95	1127	1067	1074	1067
query96	656	318	324	318
query97	6381	6379	6362	6362
query98	187	172	174	172
query99	2803	950	944	944
Total cold run time: 312858 ms
Total hot run time: 201784 ms

doris-robot · 2024-03-26T10:31:17Z

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 80b76bf41bbe72fdba24346ec3193607eea74519 with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.4 seconds inserted 10000000 Rows, about 467K ops/s

…29045 apache#28135 (apache#32839)

fix wrong result after reorder mor table
80b76bf

github-actionsBot added the kind/test label Mar 26, 2024

github-actionsBot reviewed Mar 26, 2024
View reviewed changes

BiteTheDDDDt added dev/2.0.0 2.0.0 release dev/2.0.x and removed dev/2.0.0 2.0.0 release labels Mar 27, 2024

xiaokang merged commit bce3c22 into apache:branch-2.0Apr 2, 2024

xiaokang removed the dev/2.0.x label Apr 2, 2024

cambyzju mentioned this pull request May 11, 2024
[fix](index) should not use light index change for bloom filter index #34732
Merged

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
[Bug](schame-change) fix wrong result after reorder mor tableapache#…
bde9ac4
…29045 apache#28135 (apache#32839)

BiteTheDDDDt deleted the cp_0326 branch January 20, 2025 06:58

Navigation Menu

Uh oh!

[Bug](schame-change) fix wrong result after reorder mor table - #32839

[Bug](schame-change) fix wrong result after reorder mor table#32839
xiaokang merged 1 commit into
apache:branch-2.0from
BiteTheDDDDt:cp_0326

BiteTheDDDDt commented Mar 26, 2024•
edited
Loading

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

BiteTheDDDDt commented Mar 26, 2024

Uh oh!

github-actionsBot left a comment

Uh oh!

github-actionsBotMar 26, 2024

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	boolhas_reference() const { returnexpr!=nullptr\|\|ref_column >= 0; }
	[[nodiscard]] boolhas_reference() const { returnexpr!=nullptr\|\|ref_column >= 0; }

Uh oh!

Conversation

BiteTheDDDDt commented Mar 26, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

BiteTheDDDDt commented Mar 26, 2024

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBotMar 26, 2024

Choose a reason for hiding this comment

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

doris-robot commented Mar 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

BiteTheDDDDt commented Mar 26, 2024•
edited
Loading