[Fix](warmup) one-time and periodic warm up job should be triggered regardless whether the rowset has been warmed up before by bobhan1 · Pull Request #56737 · apache/doris

bobhan1 · 2025-10-09T06:21:02Z

What problem does this PR solve?

Related PR: #54611

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-10-09T06:21:07Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

bobhan1 · 2025-10-09T09:05:01Z

run buildall

bobhan1 · 2025-10-09T09:25:54Z

run buildall

doris-robot · 2025-10-09T10:06:10Z

ClickBench: Total hot run time: 30.18 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f3efcff32e7c856098bf73f49e9ba1abc662cc91, data reload: false
query1	0.05	0.05	0.05
query2	0.09	0.06	0.05
query3	0.25	0.09	0.08
query4	1.62	0.12	0.11
query5	0.27	0.26	0.26
query6	1.19	0.65	0.65
query7	0.03	0.03	0.03
query8	0.06	0.05	0.04
query9	0.62	0.53	0.52
query10	0.57	0.59	0.57
query11	0.19	0.12	0.12
query12	0.16	0.13	0.12
query13	0.63	0.62	0.60
query14	1.03	1.06	1.03
query15	0.87	0.86	0.85
query16	0.40	0.40	0.39
query17	1.04	1.03	1.04
query18	0.21	0.19	0.20
query19	1.96	1.84	1.83
query20	0.02	0.01	0.01
query21	15.42	0.93	0.59
query22	0.76	1.24	0.64
query23	15.05	1.39	0.64
query24	7.43	1.19	0.52
query25	0.49	0.16	0.14
query26	0.65	0.16	0.14
query27	0.06	0.05	0.06
query28	10.01	1.38	0.94
query29	12.61	3.98	3.37
query30	0.28	0.14	0.13
query31	2.84	0.59	0.38
query32	3.24	0.54	0.48
query33	3.05	3.02	3.11
query34	16.10	5.47	4.82
query35	4.96	4.93	4.89
query36	0.70	0.51	0.52
query37	0.10	0.08	0.07
query38	0.07	0.05	0.05
query39	0.03	0.03	0.03
query40	0.17	0.16	0.15
query41	0.09	0.03	0.03
query42	0.04	0.04	0.03
query43	0.04	0.03	0.04
Total cold run time: 105.45 s
Total hot run time: 30.18 s

doris-robot · 2025-10-09T10:37:13Z

BE UT Coverage Report

Increment line coverage 96.88% (62/64) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.49% (17710/33741)
Line Coverage	37.66% (160782/426907)
Region Coverage	32.14% (122770/381968)
Branch Coverage	33.54% (53853/160549)

hello-stephen · 2025-10-09T11:49:58Z

BE Regression && UT Coverage Report

Increment line coverage 69.57% (64/92) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.20% (23538/33058)
Line Coverage	57.63% (245800/426490)
Region Coverage	52.77% (204107/386793)
Branch Coverage	54.58% (88062/161356)

bobhan1 · 2025-10-09T11:57:51Z

run p0

hello-stephen · 2025-10-09T13:34:47Z

BE Regression && UT Coverage Report

Increment line coverage 69.57% (64/92) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.22% (23544/33058)
Line Coverage	57.65% (245883/426490)
Region Coverage	52.76% (204088/386793)
Branch Coverage	54.60% (88100/161356)

dataroaring

LGTM

github-actions · 2025-10-10T02:32:38Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-10-10T02:32:41Z

PR approved by anyone and no changes requested.

freemandealer · 2025-10-10T06:28:11Z

+ auto existing_state = _rowset_warm_up_states[rowset_id].state;
+
+ // For job-triggered warmup (one-time and periodic warmup), allow it to proceed
+ // except when there's already another job-triggered warmup in progress


what if the job-triigered warmup is done? still return false?

freemandealer · 2025-10-10T06:33:11Z

 }
- _rowset_warm_up_states[rowset.rowset_id()] = {
- .state = state, .num_segments = rowset.num_segments(), .start_tp = start_tp};
+ _rowset_warm_up_states[rowset_id] = {


Since two different trigger types will share the same states (and the same counter in it), will this replacement of elements in the map interfere with other warmup types with the same rowsetid?

only one trigger source can update a rowset's state in any time

bobhan1 · 2025-10-10T08:40:22Z

run buildall

doris-robot · 2025-10-10T09:34:52Z

TPC-DS: Total hot run time: 189701 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6f27bc248e81ad34c30e0d2f6692734ad1cffc00, data reload: false
query1	1067	439	413	413
query2	6575	1700	1712	1700
query3	6760	224	218	218
query4	26291	24213	23224	23224
query5	5514	625	499	499
query6	360	236	225	225
query7	4661	498	301	301
query8	296	265	249	249
query9	8718	2551	2573	2551
query10	541	364	320	320
query11	15247	15019	14758	14758
query12	177	122	111	111
query13	1666	554	428	428
query14	11035	9213	9276	9213
query15	208	193	174	174
query16	7560	685	505	505
query17	1181	743	618	618
query18	2012	458	365	365
query19	241	253	191	191
query20	150	136	145	136
query21	407	188	128	128
query22	4257	4676	4731	4676
query23	35025	33715	34024	33715
query24	8671	2540	2447	2447
query25	598	509	458	458
query26	1227	274	155	155
query27	2757	510	350	350
query28	4355	2173	2139	2139
query29	809	606	498	498
query30	303	224	203	203
query31	920	828	740	740
query32	80	70	72	70
query33	579	375	327	327
query34	813	845	514	514
query35	805	832	780	780
query36	979	992	936	936
query37	123	114	99	99
query38	3598	3535	3496	3496
query39	1487	1440	1403	1403
query40	228	135	150	135
query41	67	73	58	58
query42	121	114	113	113
query43	490	513	483	483
query44	1323	828	830	828
query45	185	185	174	174
query46	831	999	629	629
query47	1753	1815	1744	1744
query48	386	415	308	308
query49	771	506	406	406
query50	645	696	408	408
query51	3927	3902	3866	3866
query52	109	116	99	99
query53	244	273	202	202
query54	603	590	527	527
query55	94	87	88	87
query56	346	339	317	317
query57	1189	1226	1143	1143
query58	334	283	282	282
query59	2592	2671	2517	2517
query60	361	341	334	334
query61	154	154	152	152
query62	805	737	680	680
query63	245	192	203	192
query64	4416	1149	852	852
query65	4014	3966	3943	3943
query66	1066	441	354	354
query67	15447	15080	15041	15041
query68	7821	930	596	596
query69	505	341	301	301
query70	1507	1355	1387	1355
query71	447	362	321	321
query72	5759	4926	4906	4906
query73	648	591	362	362
query74	8870	8913	8729	8729
query75	3513	3373	2861	2861
query76	3379	1206	758	758
query77	568	407	315	315
query78	9550	9809	8909	8909
query79	2087	826	586	586
query80	622	562	496	496
query81	495	264	231	231
query82	440	162	137	137
query83	276	270	255	255
query84	254	116	96	96
query85	866	460	431	431
query86	351	324	300	300
query87	3763	3711	3705	3705
query88	3474	2221	2239	2221
query89	397	336	305	305
query90	2038	207	213	207
query91	159	167	139	139
query92	80	69	63	63
query93	1136	974	642	642
query94	753	419	322	322
query95	405	324	316	316
query96	488	570	285	285
query97	2974	2993	2867	2867
query98	250	212	220	212
query99	1505	1411	1265	1265
Total cold run time: 276179 ms
Total hot run time: 189701 ms

doris-robot · 2025-10-10T09:40:22Z

ClickBench: Total hot run time: 30.42 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6f27bc248e81ad34c30e0d2f6692734ad1cffc00, data reload: false
query1	0.06	0.05	0.05
query2	0.09	0.05	0.05
query3	0.25	0.08	0.09
query4	1.60	0.11	0.12
query5	0.27	0.26	0.25
query6	1.18	0.65	0.64
query7	0.03	0.03	0.03
query8	0.06	0.05	0.04
query9	0.62	0.53	0.53
query10	0.58	0.58	0.57
query11	0.16	0.12	0.11
query12	0.15	0.12	0.13
query13	0.64	0.62	0.62
query14	1.00	1.04	1.02
query15	0.86	0.85	0.86
query16	0.40	0.39	0.39
query17	1.06	1.10	1.02
query18	0.21	0.20	0.20
query19	1.95	1.81	1.82
query20	0.02	0.01	0.01
query21	15.45	0.93	0.57
query22	0.78	1.20	0.61
query23	14.97	1.37	0.62
query24	7.13	1.03	0.92
query25	0.48	0.28	0.11
query26	0.63	0.15	0.15
query27	0.06	0.06	0.06
query28	9.94	1.35	0.92
query29	12.56	3.90	3.31
query30	0.28	0.14	0.11
query31	2.84	0.59	0.38
query32	3.24	0.56	0.48
query33	3.00	3.17	3.08
query34	16.22	5.47	4.84
query35	4.90	4.92	4.86
query36	0.69	0.52	0.51
query37	0.10	0.08	0.08
query38	0.06	0.04	0.04
query39	0.04	0.03	0.04
query40	0.18	0.15	0.14
query41	0.09	0.03	0.03
query42	0.05	0.03	0.03
query43	0.04	0.04	0.04
Total cold run time: 104.92 s
Total hot run time: 30.42 s

bobhan1 · 2025-10-10T11:25:07Z

run cloud_p0

bobhan1 · 2025-10-10T11:25:52Z

run nonConcurrent

doris-robot · 2025-10-10T11:27:38Z

BE UT Coverage Report

Increment line coverage 97.06% (66/68) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.51% (17720/33745)
Line Coverage	37.69% (160915/426938)
Region Coverage	32.16% (122865/382014)
Branch Coverage	33.57% (53895/160566)

hello-stephen · 2025-10-10T13:09:46Z

BE Regression && UT Coverage Report

Increment line coverage 65.62% (63/96) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.26% (18931/33062)
Line Coverage	42.47% (181139/426524)
Region Coverage	37.02% (143198/386839)
Branch Coverage	38.08% (61447/161373)

hello-stephen · 2025-10-10T13:14:12Z

BE Regression && UT Coverage Report

Increment line coverage 65.62% (63/96) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.26% (18931/33062)
Line Coverage	42.47% (181139/426524)
Region Coverage	37.02% (143198/386839)
Branch Coverage	38.08% (61447/161373)

freemandealer

LGTM

dataroaring

LGTM

github-actions · 2025-10-11T11:12:08Z

PR approved by at least one committer and no changes requested.

…egardless whether the rowset has been warmed up before (#56737) ### What problem does this PR solve? Related PR: #54611

… triggered regardless whether the rowset has been warmed up before #56737 (#56865) Cherry-picked from #56737 Co-authored-by: bobhan1 <baohan@selectdb.com>

… triggered regardless whether the rowset has been warmed up before #56737 (#56864) Cherry-picked from #56737 Co-authored-by: bobhan1 <baohan@selectdb.com>

tmp
8a3edbd

bobhan1 force-pushed the warmup-job-ignore-warmup-state branch from 4f683de to 7884d06Compare October 9, 2025 07:37

tmp
72a1afb

bobhan1 force-pushed the warmup-job-ignore-warmup-state branch from 7884d06 to 72a1afbCompare October 9, 2025 07:56

bobhan1 added 2 commits October 9, 2025 16:44

tmp2
79dc06a

format
d414ea7

add ut by claude
f3efcff

bobhan1 marked this pull request as ready for review October 9, 2025 09:25

bobhan1 requested review from dataroaring, gavinchou and w41ter as code owners October 9, 2025 09:25

dataroaring previously approved these changes Oct 10, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 10, 2025

github-actionsBot added the reviewed label Oct 10, 2025

dataroaring added dev/4.0.x dev/3.1.x labels Oct 10, 2025

freemandealer reviewed Oct 10, 2025
View reviewed changes

bobhan1 dismissed dataroaring’s stale review via 8d9b99dOctober 10, 2025 08:36

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Oct 10, 2025

fix
6f27bc2

bobhan1 force-pushed the warmup-job-ignore-warmup-state branch from 8d9b99d to 6f27bc2Compare October 10, 2025 08:38

freemandealer approved these changes Oct 11, 2025
View reviewed changes

dataroaring approved these changes Oct 11, 2025
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 11, 2025

dataroaring merged commit 00580dd into apache:masterOct 11, 2025
26 of 28 checks passed

github-actionsBot pushed a commit that referenced this pull request Oct 11, 2025
[Fix](warmup) one-time and periodic warm up job should be triggered r…
89bcd75
…egardless whether the rowset has been warmed up before (#56737) ### What problem does this PR solve? Related PR: #54611

github-actionsBot mentioned this pull request Oct 11, 2025
branch-3.1: [Fix](warmup) one-time and periodic warm up job should be triggered regardless whether the rowset has been warmed up before #56737 #56864
Merged

github-actionsBot pushed a commit that referenced this pull request Oct 11, 2025
[Fix](warmup) one-time and periodic warm up job should be triggered r…
8397350
…egardless whether the rowset has been warmed up before (#56737) ### What problem does this PR solve? Related PR: #54611

github-actionsBot mentioned this pull request Oct 11, 2025
branch-4.0: [Fix](warmup) one-time and periodic warm up job should be triggered regardless whether the rowset has been warmed up before #56737 #56865
Merged

yiguolei added dev/4.0.1-merged and removed dev/4.0.x labels Oct 12, 2025

morrySnow added dev/3.1.2-merged and removed dev/3.1.x labels Oct 13, 2025

Uh oh!

Conversation

bobhan1 commented Oct 9, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Oct 9, 2025

Uh oh!

bobhan1 commented Oct 9, 2025

Uh oh!

bobhan1 commented Oct 9, 2025

Uh oh!

doris-robot commented Oct 9, 2025

Uh oh!

doris-robot commented Oct 9, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Oct 9, 2025

BE Regression && UT Coverage Report

Uh oh!

bobhan1 commented Oct 9, 2025

Uh oh!

hello-stephen commented Oct 9, 2025

BE Regression && UT Coverage Report

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Oct 10, 2025

Uh oh!

github-actionsBot commented Oct 10, 2025

Uh oh!

freemandealerOct 10, 2025

Choose a reason for hiding this comment

Uh oh!

freemandealerOct 10, 2025

Choose a reason for hiding this comment

Uh oh!

bobhan1Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

bobhan1 commented Oct 10, 2025

Uh oh!

doris-robot commented Oct 10, 2025

Uh oh!

doris-robot commented Oct 10, 2025

Uh oh!

bobhan1 commented Oct 10, 2025

Uh oh!

bobhan1 commented Oct 10, 2025

Uh oh!

doris-robot commented Oct 10, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented Oct 10, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Oct 10, 2025

BE Regression && UT Coverage Report

Uh oh!

freemandealer left a comment

Choose a reason for hiding this comment

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Oct 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

bobhan1 commented Oct 9, 2025•
edited
Loading