[bvar](cloud-mow) Add bvar for mow compaction get delete bitmap lock backoff sleep time by bobhan1 · Pull Request #52044 · apache/doris

bobhan1 · 2025-06-20T06:46:12Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-06-20T06:46:19Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

dataroaring

LGTM

github-actions · 2025-06-20T07:03:41Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-06-20T07:03:43Z

PR approved by anyone and no changes requested.

zhannngchen

LGTM

bobhan1 · 2025-06-23T02:40:42Z

run buildall

doris-robot · 2025-06-23T03:35:37Z

TPC-H: Total hot run time: 33678 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 03d405a2422860d3cba05d9673e61e7aba7153c8, data reload: false
------ Round 1 ----------------------------------
q1	17601	5183	5068	5068
q2	1954	302	190	190
q3	10349	1258	748	748
q4	10220	1006	522	522
q5	7535	2311	2336	2311
q6	176	165	135	135
q7	899	734	610	610
q8	9325	1210	1041	1041
q9	6748	5056	5028	5028
q10	6889	2359	1956	1956
q11	474	284	269	269
q12	347	348	214	214
q13	17774	3636	3028	3028
q14	237	230	223	223
q15	569	490	491	490
q16	429	434	373	373
q17	588	867	354	354
q18	7609	7097	7147	7097
q19	2342	953	530	530
q20	320	323	222	222
q21	3652	2501	2294	2294
q22	1053	1019	975	975
Total cold run time: 107090 ms
Total hot run time: 33678 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5212	5088	5083	5083
q2	243	320	229	229
q3	2182	2636	2272	2272
q4	1385	1830	1353	1353
q5	4327	4088	4290	4088
q6	212	168	129	129
q7	2040	1918	1781	1781
q8	2534	2659	2499	2499
q9	7104	7144	7129	7129
q10	3050	3317	2828	2828
q11	569	501	491	491
q12	691	768	585	585
q13	3478	3909	3181	3181
q14	279	289	275	275
q15	525	473	480	473
q16	441	483	438	438
q17	1136	1472	1375	1375
q18	7485	7192	7037	7037
q19	758	769	807	769
q20	1932	1985	1842	1842
q21	4669	4294	4326	4294
q22	1047	1020	975	975
Total cold run time: 51299 ms
Total hot run time: 49126 ms

doris-robot · 2025-06-23T03:47:07Z

TPC-DS: Total hot run time: 185525 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 03d405a2422860d3cba05d9673e61e7aba7153c8, data reload: false
query1	986	386	396	386
query2	6520	1811	1830	1811
query3	6745	220	220	220
query4	26406	23453	23202	23202
query5	4416	627	483	483
query6	313	218	200	200
query7	4631	513	289	289
query8	271	229	214	214
query9	8611	2637	2649	2637
query10	472	321	282	282
query11	15277	15245	14711	14711
query12	173	116	108	108
query13	1659	539	421	421
query14	9496	6130	6113	6113
query15	207	202	170	170
query16	7674	625	471	471
query17	1200	737	590	590
query18	2035	408	301	301
query19	205	196	168	168
query20	122	118	115	115
query21	209	137	109	109
query22	4058	4090	4038	4038
query23	33826	33173	33257	33173
query24	8465	2339	2352	2339
query25	547	456	387	387
query26	1211	265	144	144
query27	2721	496	345	345
query28	4268	2108	2085	2085
query29	737	545	435	435
query30	283	218	191	191
query31	934	834	737	737
query32	72	66	64	64
query33	572	373	302	302
query34	781	851	537	537
query35	794	817	717	717
query36	932	987	874	874
query37	130	96	84	84
query38	4031	4114	4003	4003
query39	1469	1420	1478	1420
query40	207	120	108	108
query41	63	58	59	58
query42	124	111	115	111
query43	495	509	476	476
query44	1302	836	841	836
query45	182	168	163	163
query46	840	1016	625	625
query47	1749	1833	1741	1741
query48	385	425	308	308
query49	724	490	382	382
query50	622	687	397	397
query51	4131	4143	4087	4087
query52	108	109	97	97
query53	221	268	189	189
query54	566	565	511	511
query55	88	84	84	84
query56	319	290	282	282
query57	1189	1208	1109	1109
query58	270	262	262	262
query59	2527	2580	2522	2522
query60	345	327	319	319
query61	125	121	122	121
query62	805	730	653	653
query63	226	185	184	184
query64	4280	992	665	665
query65	4311	4164	4146	4146
query66	1064	409	322	322
query67	15682	15643	15638	15638
query68	8370	884	533	533
query69	475	301	272	272
query70	1209	1105	1067	1067
query71	460	328	319	319
query72	5211	4858	4637	4637
query73	688	577	356	356
query74	9137	8791	8917	8791
query75	3784	3181	2700	2700
query76	3610	1181	741	741
query77	786	376	292	292
query78	10022	10256	9335	9335
query79	1557	824	570	570
query80	643	519	434	434
query81	487	255	229	229
query82	433	127	95	95
query83	272	254	245	245
query84	273	107	85	85
query85	779	353	311	311
query86	338	296	289	289
query87	4338	4468	4330	4330
query88	2917	2301	2300	2300
query89	380	308	294	294
query90	1915	211	216	211
query91	142	140	113	113
query92	77	62	62	62
query93	1053	931	585	585
query94	672	425	308	308
query95	386	300	279	279
query96	488	565	279	279
query97	2727	2752	2699	2699
query98	230	206	206	206
query99	1460	1388	1286	1286
Total cold run time: 271728 ms
Total hot run time: 185525 ms

doris-robot · 2025-06-23T03:52:19Z

ClickBench: Total hot run time: 29.27 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 03d405a2422860d3cba05d9673e61e7aba7153c8, data reload: false
query1	0.04	0.04	0.03
query2	0.07	0.03	0.04
query3	0.24	0.08	0.06
query4	1.59	0.10	0.10
query5	0.41	0.43	0.40
query6	1.16	0.64	0.67
query7	0.03	0.01	0.02
query8	0.05	0.03	0.04
query9	0.58	0.52	0.51
query10	0.55	0.58	0.56
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.63	0.61	0.62
query14	0.80	0.81	0.81
query15	0.91	0.87	0.90
query16	0.40	0.38	0.39
query17	1.04	1.07	1.10
query18	0.23	0.21	0.21
query19	2.02	1.87	1.84
query20	0.01	0.01	0.01
query21	15.42	0.91	0.55
query22	0.76	1.13	0.84
query23	14.74	1.42	0.64
query24	7.24	1.11	0.34
query25	0.41	0.20	0.14
query26	0.69	0.17	0.15
query27	0.07	0.05	0.05
query28	8.97	0.94	0.44
query29	12.56	4.05	3.35
query30	0.26	0.10	0.06
query31	2.82	0.60	0.40
query32	3.24	0.55	0.47
query33	3.09	3.02	3.14
query34	16.15	5.40	4.82
query35	4.81	4.83	4.79
query36	0.69	0.52	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.18	0.14	0.15
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 103.48 s
Total hot run time: 29.27 s

hello-stephen · 2025-06-23T04:43:25Z

BE UT Coverage Report

Increment line coverage 0.00% (0/5) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	56.55% (15099/26702)
Line Coverage	45.32% (135253/298427)
Region Coverage	44.42% (67980/153039)
Branch Coverage	38.99% (34898/89502)

hello-stephen · 2025-06-23T05:09:56Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/5) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	60.06% (15778/26269)
Line Coverage	49.25% (146888/298226)
Region Coverage	46.45% (83625/180013)
Branch Coverage	39.83% (40737/102270)

hello-stephen · 2025-06-23T05:22:47Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/5) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	60.06% (15778/26269)
Line Coverage	49.25% (146888/298226)
Region Coverage	46.45% (83625/180013)
Branch Coverage	39.83% (40737/102270)

bobhan1 · 2025-06-23T09:13:53Z

run p0

bobhan1 · 2025-06-23T09:13:58Z

run cloud_p0

hello-stephen · 2025-06-23T11:20:09Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/5) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	62.68% (16465/26269)
Line Coverage	52.11% (155398/298226)
Region Coverage	49.62% (89331/180013)
Branch Coverage	42.99% (43967/102270)

dataroaring

LGTM

github-actions · 2025-06-23T13:45:19Z

PR approved by at least one committer and no changes requested.

koarz

LGTM

zhannngchen

LGTM

…backoff sleep time (#52044)

…backoff sleep time (apache#52044)

…bitmap lock backoff sleep time #52044 (#52289) Cherry-picked from #52044 Co-authored-by: bobhan1 <baohan@selectdb.com>

…bitmap lock backoff sleep time (#52044) (#52297) pick #52044

…bitmap lock backoff sleep time (apache#52044) (apache#52297) pick apache#52044

bobhan1 requested review from dataroaring and gavinchou as code owners June 20, 2025 06:46

dataroaring previously approved these changes Jun 20, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 20, 2025

github-actionsBot added the reviewed label Jun 20, 2025

zhannngchen previously approved these changes Jun 20, 2025
View reviewed changes

update
03d405a

bobhan1 dismissed stale reviews from zhannngchen and dataroaring via 03d405aJune 23, 2025 01:58

bobhan1 force-pushed the recored-be-get-mow-lock-backoff-sleep-time branch from 626efeb to 03d405aCompare June 23, 2025 01:58

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Jun 23, 2025

This was referenced Jun 23, 2025
branch-3.0: [Opt](cloud-mow) Add more delete bitmap verbose log (#51751) #52121
Merged
branch-3.1: [Opt](cloud-mow) Add more delete bitmap verbose log (#51751) #52122
Merged

dataroaring approved these changes Jun 23, 2025
View reviewed changes

dataroaring added dev/3.0.x dev/3.1.x labels Jun 23, 2025

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 23, 2025

koarz approved these changes Jun 25, 2025
View reviewed changes

zhannngchen approved these changes Jun 25, 2025
View reviewed changes

zhannngchen merged commit 2bae60a into apache:masterJun 25, 2025

github-actionsBot added the dev/3.0.x-conflict label Jun 25, 2025

github-actionsBot pushed a commit that referenced this pull request Jun 25, 2025
[bvar](cloud-mow) Add bvar for mow compaction get delete bitmap lock …
4dd91d4
…backoff sleep time (#52044)

github-actionsBot mentioned this pull request Jun 25, 2025
branch-3.1: [bvar](cloud-mow) Add bvar for mow compaction get delete bitmap lock backoff sleep time #52044 #52289
Merged

bobhan1 added a commit to bobhan1/doris that referenced this pull request Jun 25, 2025
[bvar](cloud-mow) Add bvar for mow compaction get delete bitmap lock …
742713d
…backoff sleep time (apache#52044)

bobhan1 mentioned this pull request Jun 25, 2025
branch-3.0: [bvar](cloud-mow) Add bvar for mow compaction get delete bitmap lock backoff sleep time (#52044) #52297
Merged

morrySnow pushed a commit that referenced this pull request Jun 26, 2025
branch-3.1: [bvar](cloud-mow) Add bvar for mow compaction get delete …
3a1b09e
…bitmap lock backoff sleep time #52044 (#52289) Cherry-picked from #52044 Co-authored-by: bobhan1 <baohan@selectdb.com>

morrySnow added dev/3.1.0-merged and removed dev/3.1.x labels Jun 26, 2025

dataroaring pushed a commit that referenced this pull request Jun 27, 2025
branch-3.0: [bvar](cloud-mow) Add bvar for mow compaction get delete …
a425e42
…bitmap lock backoff sleep time (#52044) (#52297) pick #52044

dataroaring added dev/3.0.7-merged and removed dev/3.0.x dev/3.0.x-conflict labels Jun 27, 2025

koarz pushed a commit to koarz/doris that referenced this pull request Jul 3, 2025
branch-3.0: [bvar](cloud-mow) Add bvar for mow compaction get delete …
767394f
…bitmap lock backoff sleep time (apache#52044) (apache#52297) pick apache#52044

gavinchou mentioned this pull request Aug 15, 2025
Release Note 3.0.7 #54868
Closed

Uh oh!

Conversation

bobhan1 commented Jun 20, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Jun 20, 2025

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jun 20, 2025

Uh oh!

github-actionsBot commented Jun 20, 2025

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

bobhan1 commented Jun 23, 2025

Uh oh!

doris-robot commented Jun 23, 2025

Uh oh!

doris-robot commented Jun 23, 2025

Uh oh!

doris-robot commented Jun 23, 2025

Uh oh!

hello-stephen commented Jun 23, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Jun 23, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Jun 23, 2025

BE Regression && UT Coverage Report

Uh oh!

bobhan1 commented Jun 23, 2025

Uh oh!

bobhan1 commented Jun 23, 2025

Uh oh!

hello-stephen commented Jun 23, 2025

BE Regression && UT Coverage Report

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jun 23, 2025

Uh oh!

koarz left a comment

Choose a reason for hiding this comment

Uh oh!

zhannngchen left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants