[feat](job)Internal job cancellation immediately and the strong association with the STARTS parameter by CalvinKirs · Pull Request #36805 · apache/doris

CalvinKirs · 2024-06-25T08:45:43Z

Proposed changes

For internal tasks, such as MTMV, the start time may already be set, or the time may be adjusted immediately.

doris-robot · 2024-06-25T08:45:48Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

…iation with the STARTS parameter For internal tasks, such as MTMV, the start time may already be set, or the time may be adjusted immediately.

CalvinKirs · 2024-06-26T06:21:36Z

run buildall

doris-robot · 2024-06-26T07:27:06Z

TPC-H: Total hot run time: 40447 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d298cb4bf70c6430d69e4f2e3f4144ce7ccae414, data reload: false
------ Round 1 ----------------------------------
q1	18245	4638	5030	4638
q2	2012	181	186	181
q3	10538	1154	1147	1147
q4	10188	805	794	794
q5	7469	2738	2690	2690
q6	225	136	136	136
q7	947	597	602	597
q8	9222	2124	2066	2066
q9	9119	6550	6503	6503
q10	8901	3771	3743	3743
q11	474	251	248	248
q12	406	234	239	234
q13	17868	3033	3019	3019
q14	279	228	235	228
q15	503	484	486	484
q16	526	382	377	377
q17	976	780	721	721
q18	8175	7566	7390	7390
q19	8157	1504	1402	1402
q20	664	313	324	313
q21	4930	3202	4022	3202
q22	399	346	334	334
Total cold run time: 120223 ms
Total hot run time: 40447 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4516	4331	4284	4284
q2	381	265	257	257
q3	2989	2790	2736	2736
q4	1899	1614	1614	1614
q5	5247	5277	5302	5277
q6	214	128	130	128
q7	2115	1741	1674	1674
q8	3205	3366	3362	3362
q9	8405	8340	8386	8340
q10	3962	3709	3730	3709
q11	616	480	477	477
q12	775	611	588	588
q13	17481	2971	2991	2971
q14	308	255	271	255
q15	533	473	485	473
q16	464	420	435	420
q17	1789	1468	1516	1468
q18	7597	7604	7312	7312
q19	1719	1643	1538	1538
q20	2007	1788	1777	1777
q21	4926	4663	4649	4649
q22	626	587	545	545
Total cold run time: 71774 ms
Total hot run time: 53854 ms

doris-robot · 2024-06-26T07:38:33Z

TPC-DS: Total hot run time: 174849 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d298cb4bf70c6430d69e4f2e3f4144ce7ccae414, data reload: false
query1	915	383	370	370
query2	6429	2415	2365	2365
query3	6638	199	199	199
query4	19832	17612	17584	17584
query5	4124	468	468	468
query6	245	168	177	168
query7	4591	307	295	295
query8	324	305	307	305
query9	8490	2448	2431	2431
query10	606	281	283	281
query11	10647	10232	10149	10149
query12	139	84	80	80
query13	1624	369	372	369
query14	10051	7588	7579	7579
query15	248	192	191	191
query16	7888	286	275	275
query17	1879	547	559	547
query18	2014	282	282	282
query19	208	161	169	161
query20	93	85	88	85
query21	225	136	124	124
query22	4425	4186	4094	4094
query23	33811	33005	33120	33005
query24	11129	2912	2756	2756
query25	661	393	395	393
query26	1641	153	159	153
query27	2972	318	320	318
query28	7096	2108	2089	2089
query29	1019	648	641	641
query30	285	152	154	152
query31	971	733	762	733
query32	94	55	55	55
query33	791	312	290	290
query34	912	476	467	467
query35	774	645	642	642
query36	1084	941	965	941
query37	158	75	71	71
query38	2910	2771	2775	2771
query39	876	804	815	804
query40	285	130	127	127
query41	59	53	53	53
query42	123	97	99	97
query43	596	529	528	528
query44	1223	741	743	741
query45	200	161	167	161
query46	1080	738	750	738
query47	1875	1808	1787	1787
query48	374	298	297	297
query49	1217	432	418	418
query50	788	387	384	384
query51	6876	6830	6783	6783
query52	108	96	98	96
query53	364	297	293	293
query54	1013	473	448	448
query55	77	77	76	76
query56	331	281	295	281
query57	1174	1082	1097	1082
query58	275	257	258	257
query59	3401	3136	2992	2992
query60	338	304	304	304
query61	128	125	106	106
query62	647	436	440	436
query63	328	288	398	288
query64	9956	2220	1790	1790
query65	3177	3131	3147	3131
query66	1389	346	334	334
query67	15807	15003	15049	15003
query68	8518	546	555	546
query69	705	478	437	437
query70	1187	1083	1042	1042
query71	515	274	275	274
query72	8816	5767	5705	5705
query73	1193	332	323	323
query74	5938	5532	5464	5464
query75	5191	2657	2636	2636
query76	4944	1022	902	902
query77	769	336	299	299
query78	10594	9917	9768	9768
query79	8190	529	513	513
query80	1015	472	460	460
query81	554	218	224	218
query82	759	112	102	102
query83	339	175	165	165
query84	275	84	89	84
query85	1359	282	264	264
query86	407	315	309	309
query87	3300	3101	3111	3101
query88	4898	2481	2471	2471
query89	534	390	387	387
query90	1997	193	183	183
query91	127	98	98	98
query92	64	50	50	50
query93	6390	513	507	507
query94	1236	189	188	188
query95	407	313	314	313
query96	613	269	271	269
query97	3215	3015	3064	3015
query98	224	197	189	189
query99	1246	853	854	853
Total cold run time: 297547 ms
Total hot run time: 174849 ms

doris-robot · 2024-06-26T07:44:03Z

ClickBench: Total hot run time: 30.7 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d298cb4bf70c6430d69e4f2e3f4144ce7ccae414, data reload: false
query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.21	0.05	0.06
query4	1.69	0.08	0.09
query5	0.51	0.49	0.49
query6	1.13	0.72	0.74
query7	0.02	0.02	0.01
query8	0.05	0.05	0.04
query9	0.55	0.49	0.49
query10	0.54	0.55	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.59
query14	0.77	0.77	0.81
query15	0.84	0.81	0.82
query16	0.36	0.37	0.37
query17	1.03	0.97	1.02
query18	0.26	0.21	0.25
query19	1.81	1.82	1.73
query20	0.01	0.01	0.01
query21	15.45	0.64	0.63
query22	4.28	7.50	1.96
query23	18.30	1.33	1.34
query24	2.10	0.21	0.22
query25	0.14	0.09	0.08
query26	0.26	0.17	0.18
query27	0.08	0.08	0.08
query28	13.29	1.03	0.99
query29	12.61	3.27	3.26
query30	0.25	0.06	0.06
query31	2.86	0.39	0.39
query32	3.30	0.48	0.46
query33	2.89	2.94	2.92
query34	17.03	4.42	4.43
query35	4.49	4.52	4.50
query36	0.64	0.51	0.46
query37	0.18	0.16	0.15
query38	0.15	0.16	0.16
query39	0.04	0.04	0.03
query40	0.17	0.13	0.15
query41	0.09	0.05	0.05
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.46 s
Total hot run time: 30.7 s

zddr

add case?

zddr · 2024-06-26T10:32:02Z

 if (startsTimeStamp.equalsIgnoreCase(CURRENT_TIMESTAMP_STRING)) {
 jobExecutionConfiguration.setImmediate(true);
+ //To avoid immediate re-scheduling, set the start time of the timer 100ms before the current time.
+ timerDefinition.setStartTimeMs(System.currentTimeMillis() - 100L);


what will happen if not set startTimemMs?

The syntax for Create Job doesn't immediately schedule the job. For a one-time schedule, setting the start time to CURRENT_TIMESTAMP achieves immediate scheduling. Therefore, we still need to set the startTime parameter; otherwise, the job query will lack a start time.

zddr · 2024-06-26T10:38:18Z

 schedulerInstantJob(job, TaskType.SCHEDULED, null);
 }
 }
+ if (job.getJobConfig().isImmediate() && JobExecuteType.ONE_TIME.equals(job.getJobConfig().getExecuteType())) {


why JobExecuteType == ONE_TIME ,can not setLatestSchedulerTimeMs(System.currentTimeMillis())?

This variable is not persistent, so for a one-time, immediately scheduled task, there's no need to set it.

github-actions · 2024-06-27T04:45:23Z

PR approved by anyone and no changes requested.

CalvinKirs · 2024-06-27T06:08:17Z

run buildall

doris-robot · 2024-06-27T06:25:42Z

TPC-H: Total hot run time: 40034 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 834394b2d65a3cf75c10a52dcd56418495a7d717, data reload: false
------ Round 1 ----------------------------------
q1	17998	4441	4392	4392
q2	2251	191	192	191
q3	10476	1189	1162	1162
q4	10193	832	838	832
q5	7469	2625	2708	2625
q6	215	133	132	132
q7	942	594	608	594
q8	9227	2078	2044	2044
q9	8966	6499	6471	6471
q10	8976	3708	3724	3708
q11	469	239	235	235
q12	398	220	226	220
q13	17774	3020	2999	2999
q14	274	220	210	210
q15	523	483	484	483
q16	523	377	383	377
q17	958	719	727	719
q18	8050	7436	7429	7429
q19	7917	1517	1420	1420
q20	664	311	326	311
q21	4849	3144	4019	3144
q22	404	342	336	336
Total cold run time: 119516 ms
Total hot run time: 40034 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4345	4265	4238	4238
q2	368	274	286	274
q3	3014	2803	2731	2731
q4	1886	1627	1615	1615
q5	5221	5268	5252	5252
q6	218	128	128	128
q7	2099	1766	1757	1757
q8	3192	3361	3348	3348
q9	8339	8329	8302	8302
q10	3868	3656	3676	3656
q11	567	490	484	484
q12	793	657	621	621
q13	17473	3015	2958	2958
q14	298	264	245	245
q15	530	474	480	474
q16	468	418	425	418
q17	1784	1480	1471	1471
q18	7672	7599	7421	7421
q19	1720	1520	1539	1520
q20	2008	1751	1750	1750
q21	4925	4890	4789	4789
q22	604	573	545	545
Total cold run time: 71392 ms
Total hot run time: 53997 ms

doris-robot · 2024-06-27T06:36:49Z

TPC-DS: Total hot run time: 173127 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 834394b2d65a3cf75c10a52dcd56418495a7d717, data reload: false
query1	916	390	376	376
query2	6462	2446	2455	2446
query3	6641	213	224	213
query4	18944	17727	17252	17252
query5	4221	487	468	468
query6	292	180	182	180
query7	4596	284	286	284
query8	335	282	300	282
query9	8497	2334	2322	2322
query10	610	293	272	272
query11	10480	10362	10050	10050
query12	130	86	83	83
query13	1652	362	361	361
query14	10051	7058	6852	6852
query15	227	194	185	185
query16	7765	274	269	269
query17	1409	547	532	532
query18	1876	292	283	283
query19	196	153	157	153
query20	92	82	84	82
query21	231	162	130	130
query22	4460	4169	4220	4169
query23	33818	33171	33139	33139
query24	12221	2820	2911	2820
query25	661	360	363	360
query26	1780	149	155	149
query27	2955	309	315	309
query28	7579	2022	2004	2004
query29	1103	620	600	600
query30	273	152	147	147
query31	956	720	748	720
query32	93	56	52	52
query33	767	284	285	284
query34	969	470	473	470
query35	715	628	612	612
query36	1097	962	949	949
query37	216	71	73	71
query38	2878	2719	2760	2719
query39	851	792	800	792
query40	289	123	124	123
query41	53	54	55	54
query42	124	98	104	98
query43	586	548	568	548
query44	1219	759	726	726
query45	201	161	164	161
query46	1074	702	714	702
query47	1876	1805	1777	1777
query48	369	299	300	299
query49	1187	403	409	403
query50	764	387	394	387
query51	6907	6835	6803	6803
query52	102	97	95	95
query53	353	289	300	289
query54	940	441	434	434
query55	80	75	74	74
query56	284	261	269	261
query57	1184	1082	1046	1046
query58	262	252	238	238
query59	3712	3025	3175	3025
query60	296	290	290	290
query61	113	111	112	111
query62	667	438	469	438
query63	336	293	295	293
query64	9938	2207	1692	1692
query65	3277	3072	3107	3072
query66	1371	347	334	334
query67	15401	15186	15051	15051
query68	5623	527	542	527
query69	613	429	363	363
query70	1129	1117	1100	1100
query71	462	268	271	268
query72	7628	6031	5369	5369
query73	772	330	330	330
query74	5875	5550	5621	5550
query75	3773	2641	2655	2641
query76	3626	970	911	911
query77	628	312	297	297
query78	10486	9885	9810	9810
query79	2543	522	501	501
query80	1086	474	457	457
query81	585	217	214	214
query82	1422	108	99	99
query83	261	169	170	169
query84	231	88	86	86
query85	1580	343	270	270
query86	475	320	338	320
query87	3255	3119	3062	3062
query88	4081	2447	2439	2439
query89	468	375	377	375
query90	1790	196	187	187
query91	127	98	96	96
query92	59	48	49	48
query93	2268	499	494	494
query94	1228	190	187	187
query95	412	311	309	309
query96	596	279	268	268
query97	3308	3095	3078	3078
query98	212	198	195	195
query99	1266	844	842	842
Total cold run time: 279848 ms
Total hot run time: 173127 ms

doris-robot · 2024-06-27T06:42:20Z

ClickBench: Total hot run time: 30.48 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 834394b2d65a3cf75c10a52dcd56418495a7d717, data reload: false
query1	0.04	0.03	0.03
query2	0.09	0.04	0.04
query3	0.23	0.06	0.05
query4	1.70	0.09	0.09
query5	0.58	0.48	0.47
query6	1.14	0.73	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.56	0.49	0.48
query10	0.55	0.55	0.54
query11	0.15	0.11	0.12
query12	0.15	0.11	0.13
query13	0.60	0.59	0.61
query14	0.78	0.81	0.79
query15	0.83	0.81	0.81
query16	0.37	0.38	0.37
query17	1.02	1.00	1.00
query18	0.23	0.24	0.26
query19	1.90	1.79	1.81
query20	0.01	0.01	0.01
query21	15.46	0.75	0.66
query22	4.58	7.09	1.75
query23	18.32	1.40	1.27
query24	2.14	0.23	0.23
query25	0.16	0.08	0.09
query26	0.26	0.17	0.17
query27	0.07	0.08	0.08
query28	13.28	1.03	1.00
query29	12.61	3.27	3.28
query30	0.26	0.07	0.06
query31	2.91	0.40	0.38
query32	3.24	0.48	0.48
query33	2.88	2.83	2.94
query34	17.19	4.39	4.38
query35	4.52	4.51	4.47
query36	0.64	0.46	0.49
query37	0.18	0.16	0.16
query38	0.15	0.14	0.15
query39	0.04	0.04	0.03
query40	0.17	0.16	0.16
query41	0.10	0.04	0.05
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.26 s
Total hot run time: 30.48 s

CalvinKirs · 2024-07-01T06:14:23Z

run buildall

doris-robot · 2024-07-01T06:59:36Z

TPC-H: Total hot run time: 39467 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7761908d488ed3932286729ba284f42a233c02c8, data reload: false
------ Round 1 ----------------------------------
q1	17613	4286	4222	4222
q2	2016	193	187	187
q3	10479	1253	1061	1061
q4	10192	797	693	693
q5	7492	2630	2552	2552
q6	221	138	136	136
q7	951	614	596	596
q8	9228	2060	2032	2032
q9	8858	6490	6480	6480
q10	9032	3683	3694	3683
q11	457	237	235	235
q12	571	236	224	224
q13	17765	2999	2959	2959
q14	263	218	210	210
q15	527	481	466	466
q16	514	383	374	374
q17	956	576	546	546
q18	8053	7462	7423	7423
q19	6133	1446	1550	1446
q20	656	324	325	324
q21	5054	3275	3389	3275
q22	391	343	343	343
Total cold run time: 117422 ms
Total hot run time: 39467 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4358	4188	4189	4188
q2	371	265	271	265
q3	3006	2839	2910	2839
q4	1979	1714	1664	1664
q5	5667	5499	5450	5450
q6	219	131	127	127
q7	2202	1840	1829	1829
q8	3289	3407	3440	3407
q9	8758	8639	8811	8639
q10	4104	3787	3705	3705
q11	611	488	498	488
q12	827	669	617	617
q13	15976	3126	3165	3126
q14	320	281	272	272
q15	520	499	493	493
q16	476	429	414	414
q17	1813	1502	1514	1502
q18	8048	7929	7709	7709
q19	1753	1640	1723	1640
q20	2213	1907	1875	1875
q21	5066	4911	4776	4776
q22	702	549	583	549
Total cold run time: 72278 ms
Total hot run time: 55574 ms

doris-robot · 2024-07-01T07:10:32Z

TPC-DS: Total hot run time: 173614 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7761908d488ed3932286729ba284f42a233c02c8, data reload: false
query1	916	381	378	378
query2	6480	2401	2448	2401
query3	6645	203	208	203
query4	19233	17683	17332	17332
query5	3662	471	487	471
query6	277	180	165	165
query7	4585	296	294	294
query8	311	302	310	302
query9	8383	2383	2359	2359
query10	610	308	298	298
query11	10543	10198	10152	10152
query12	117	82	82	82
query13	1641	359	363	359
query14	9401	6829	6775	6775
query15	235	182	181	181
query16	7263	266	261	261
query17	1444	537	524	524
query18	1945	269	276	269
query19	186	156	147	147
query20	89	82	78	78
query21	208	137	139	137
query22	4622	4317	4260	4260
query23	33855	33587	33555	33555
query24	11234	2946	2930	2930
query25	648	384	375	375
query26	1294	152	158	152
query27	2424	318	326	318
query28	7452	2110	2113	2110
query29	943	659	636	636
query30	262	164	160	160
query31	1008	760	742	742
query32	96	57	58	57
query33	771	319	298	298
query34	925	511	495	495
query35	754	643	644	643
query36	1132	980	953	953
query37	141	75	80	75
query38	2969	2832	2803	2803
query39	900	870	824	824
query40	217	125	120	120
query41	52	51	58	51
query42	114	104	97	97
query43	588	539	553	539
query44	1195	721	725	721
query45	191	164	156	156
query46	1081	687	706	687
query47	1922	1802	1780	1780
query48	365	291	282	282
query49	852	404	421	404
query50	776	379	384	379
query51	6934	6873	6788	6788
query52	111	91	90	90
query53	354	288	283	283
query54	919	479	438	438
query55	87	74	73	73
query56	282	255	263	255
query57	1113	1036	1075	1036
query58	246	249	247	247
query59	3459	3116	3274	3116
query60	303	281	276	276
query61	95	96	93	93
query62	617	457	433	433
query63	324	288	287	287
query64	8981	2274	1843	1843
query65	3218	3087	3099	3087
query66	835	339	340	339
query67	15605	15255	14868	14868
query68	4535	527	526	526
query69	588	430	329	329
query70	1034	1021	1166	1021
query71	417	286	292	286
query72	7156	5004	5375	5004
query73	738	325	319	319
query74	5872	5524	5492	5492
query75	3315	2661	2695	2661
query76	2430	992	888	888
query77	730	285	299	285
query78	10445	10036	9820	9820
query79	1904	514	502	502
query80	2296	465	475	465
query81	563	222	230	222
query82	1052	105	104	104
query83	289	192	170	170
query84	258	85	93	85
query85	1148	282	287	282
query86	437	344	331	331
query87	3294	3121	3041	3041
query88	3002	2363	2366	2363
query89	464	376	370	370
query90	1672	185	184	184
query91	132	101	102	101
query92	61	50	49	49
query93	1703	499	491	491
query94	1108	182	242	182
query95	415	321	323	321
query96	587	268	273	268
query97	3262	3077	3107	3077
query98	220	204	193	193
query99	1151	860	862	860
Total cold run time: 268718 ms
Total hot run time: 173614 ms

doris-robot · 2024-07-01T07:16:03Z

ClickBench: Total hot run time: 30.55 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7761908d488ed3932286729ba284f42a233c02c8, data reload: false
query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.04
query4	1.67	0.07	0.10
query5	0.50	0.50	0.49
query6	1.14	0.74	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.56	0.49	0.48
query10	0.55	0.54	0.54
query11	0.16	0.11	0.12
query12	0.14	0.12	0.11
query13	0.60	0.59	0.60
query14	0.79	0.78	0.78
query15	0.84	0.82	0.81
query16	0.34	0.36	0.35
query17	1.02	1.01	0.99
query18	0.20	0.28	0.24
query19	1.76	1.69	1.81
query20	0.01	0.01	0.00
query21	15.44	0.76	0.66
query22	4.10	7.33	2.01
query23	18.28	1.41	1.25
query24	2.12	0.22	0.21
query25	0.15	0.10	0.07
query26	0.27	0.17	0.18
query27	0.08	0.09	0.08
query28	13.30	1.01	1.00
query29	12.63	3.29	3.33
query30	0.26	0.06	0.06
query31	2.86	0.39	0.38
query32	3.27	0.46	0.47
query33	2.84	2.93	2.85
query34	17.24	4.39	4.41
query35	4.54	4.45	4.51
query36	0.65	0.47	0.48
query37	0.18	0.15	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.17	0.14	0.15
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.48 s
Total hot run time: 30.55 s

CalvinKirs · 2024-07-02T02:08:19Z

run buildall

doris-robot · 2024-07-02T03:35:47Z

TPC-H: Total hot run time: 39815 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7761908d488ed3932286729ba284f42a233c02c8, data reload: false
------ Round 1 ----------------------------------
q1	17603	4918	4287	4287
q2	2028	192	188	188
q3	10525	1195	1112	1112
q4	10216	784	793	784
q5	7471	2716	2667	2667
q6	228	140	136	136
q7	958	597	602	597
q8	9223	2093	2079	2079
q9	8721	6459	6503	6459
q10	8975	3735	3749	3735
q11	455	242	239	239
q12	457	235	231	231
q13	17783	3018	3008	3008
q14	269	248	224	224
q15	537	475	474	474
q16	510	385	381	381
q17	976	712	638	638
q18	8109	7598	7372	7372
q19	4741	1344	1447	1344
q20	662	313	324	313
q21	4980	3219	3910	3219
q22	408	328	333	328
Total cold run time: 115835 ms
Total hot run time: 39815 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4371	4222	4202	4202
q2	357	279	274	274
q3	3029	2892	2868	2868
q4	2013	1681	1706	1681
q5	5662	5500	5514	5500
q6	219	136	137	136
q7	2182	1852	1909	1852
q8	3257	3421	3445	3421
q9	8780	8677	8915	8677
q10	4059	3951	3661	3661
q11	563	511	483	483
q12	803	650	646	646
q13	17103	3119	3230	3119
q14	314	285	285	285
q15	524	491	493	491
q16	508	466	422	422
q17	1838	1510	1487	1487
q18	8123	8050	7650	7650
q19	1727	1664	1607	1607
q20	2081	1865	1855	1855
q21	5238	4934	4837	4837
q22	611	545	576	545
Total cold run time: 73362 ms
Total hot run time: 55699 ms

doris-robot · 2024-07-02T03:47:04Z

TPC-DS: Total hot run time: 173584 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7761908d488ed3932286729ba284f42a233c02c8, data reload: false
query1	914	377	377	377
query2	6457	2533	2404	2404
query3	6626	211	213	211
query4	18964	17616	17359	17359
query5	3643	492	482	482
query6	273	162	164	162
query7	4593	306	305	305
query8	303	299	286	286
query9	8614	2421	2375	2375
query10	570	307	282	282
query11	10342	10048	9984	9984
query12	115	88	82	82
query13	1658	371	384	371
query14	10098	7775	6866	6866
query15	258	187	190	187
query16	8040	279	281	279
query17	1858	539	562	539
query18	2104	280	274	274
query19	207	155	157	155
query20	93	83	83	83
query21	213	150	137	137
query22	4249	3974	3975	3974
query23	33708	33591	33677	33591
query24	10316	2926	2856	2856
query25	610	410	394	394
query26	706	163	159	159
query27	2227	323	324	323
query28	5893	2171	2137	2137
query29	896	671	639	639
query30	259	162	156	156
query31	973	780	778	778
query32	107	59	56	56
query33	675	314	306	306
query34	863	480	476	476
query35	748	656	623	623
query36	1127	966	996	966
query37	143	82	84	82
query38	2931	2828	2802	2802
query39	893	842	844	842
query40	214	137	137	137
query41	59	53	57	53
query42	113	98	106	98
query43	604	568	553	553
query44	1047	740	730	730
query45	202	170	169	169
query46	1069	714	753	714
query47	1857	1759	1736	1736
query48	369	317	307	307
query49	860	515	408	408
query50	759	383	394	383
query51	6912	6790	6795	6790
query52	111	89	93	89
query53	365	287	282	282
query54	850	440	446	440
query55	75	72	73	72
query56	296	263	270	263
query57	1114	1014	1061	1014
query58	236	239	237	237
query59	3530	3227	3311	3227
query60	310	286	285	285
query61	95	90	95	90
query62	619	431	418	418
query63	312	288	285	285
query64	8418	2271	1775	1775
query65	3156	3082	3099	3082
query66	738	321	324	321
query67	15765	14991	14816	14816
query68	8664	561	546	546
query69	772	489	339	339
query70	1436	1066	1135	1066
query71	505	277	272	272
query72	8778	5294	5391	5294
query73	2262	326	323	323
query74	5869	5455	5467	5455
query75	5196	2666	2660	2660
query76	5297	898	952	898
query77	754	297	338	297
query78	10473	9776	9781	9776
query79	8390	528	522	522
query80	1022	467	474	467
query81	554	221	223	221
query82	515	105	104	104
query83	337	172	168	168
query84	267	86	86	86
query85	942	280	273	273
query86	372	332	296	296
query87	3326	3032	3100	3032
query88	3978	2414	2361	2361
query89	541	380	390	380
query90	1976	188	189	188
query91	129	102	98	98
query92	65	51	48	48
query93	6495	515	507	507
query94	1305	182	183	182
query95	400	309	308	308
query96	614	265	266	265
query97	3159	2979	3017	2979
query98	209	202	196	196
query99	1129	843	860	843
Total cold run time: 289053 ms
Total hot run time: 173584 ms

doris-robot · 2024-07-02T03:52:36Z

ClickBench: Total hot run time: 31.26 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7761908d488ed3932286729ba284f42a233c02c8, data reload: false
query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.05	0.05
query4	1.65	0.08	0.11
query5	0.50	0.49	0.49
query6	1.12	0.73	0.73
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.50	0.47
query10	0.55	0.53	0.54
query11	0.16	0.12	0.12
query12	0.15	0.13	0.13
query13	0.59	0.59	0.58
query14	0.80	0.77	0.78
query15	0.85	0.82	0.82
query16	0.37	0.37	0.37
query17	1.04	1.03	1.05
query18	0.23	0.23	0.26
query19	1.82	1.70	1.66
query20	0.01	0.01	0.01
query21	15.47	0.75	0.66
query22	3.73	6.93	2.63
query23	18.25	1.41	1.27
query24	2.16	0.23	0.22
query25	0.15	0.08	0.08
query26	0.27	0.17	0.17
query27	0.09	0.08	0.08
query28	13.30	1.03	1.00
query29	12.66	3.25	3.22
query30	0.26	0.06	0.05
query31	2.85	0.39	0.39
query32	3.28	0.49	0.48
query33	2.89	2.94	2.90
query34	17.08	4.42	4.42
query35	4.51	4.46	4.48
query36	0.64	0.46	0.47
query37	0.19	0.16	0.16
query38	0.15	0.16	0.14
query39	0.04	0.03	0.03
query40	0.20	0.14	0.13
query41	0.09	0.05	0.05
query42	0.06	0.04	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.16 s
Total hot run time: 31.26 s

CalvinKirs · 2024-07-03T06:02:40Z

run external

JNSimba

LGTM

github-actions · 2024-07-03T07:46:14Z

PR approved by at least one committer and no changes requested.

…iation with the STARTS parameter (#36805) ## Proposed changes For internal tasks, such as MTMV, the start time may already be set, or the time may be adjusted immediately.

…iation with the STARTS parameter (apache#36805) ## Proposed changes For internal tasks, such as MTMV, the start time may already be set, or the time may be adjusted immediately.  (cherry picked from commit 904a6c0)

extends: #36805 Previously, if the user set immediate execution, the current time would replace the user's set start time

extends: apache#36805 Previously, if the user set immediate execution, the current time would replace the user's set start time

…iation with the STARTS parameter (#36805) (#38110) … ## Proposed changes For internal tasks, such as MTMV, the start time may already be set, or the time may be adjusted immediately.  (cherry picked from commit 904a6c0) ## Proposed changes Issue Number: close#36805

extends: #36805 Previously, if the user set immediate execution, the current time would replace the user's set start time

…iation with the STARTS parameter (apache#36805) ## Proposed changes For internal tasks, such as MTMV, the start time may already be set, or the time may be adjusted immediately.

extends: apache#36805 Previously, if the user set immediate execution, the current time would replace the user's set start time

[feat](job)Internal job cancellation immediately and the strong assoc…
d298cb4
…iation with the STARTS parameter For internal tasks, such as MTMV, the start time may already be set, or the time may be adjusted immediately.

CalvinKirs force-pushed the master-job-set-starttime branch from 082863c to d298cb4Compare June 25, 2024 08:46

zddr reviewed Jun 26, 2024
View reviewed changes

add ut
834394b

zddr approved these changes Jun 27, 2024
View reviewed changes

github-actionsBot added the reviewed label Jun 27, 2024

CalvinKirs added the dev/2.1.x label Jun 27, 2024

Merge branch 'master' into master-job-set-starttime
7761908

JNSimba approved these changes Jul 3, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jul 3, 2024

CalvinKirs merged commit 904a6c0 into apache:masterJul 10, 2024

CalvinKirs deleted the master-job-set-starttime branch July 10, 2024 02:08

CalvinKirs mentioned this pull request Jul 19, 2024
[feat](job)Internal job cancellation immediately and the strong association with the STARTS parameter (#36805) #38110
Merged

dataroaring added the dev/3.0.1-merged label Aug 5, 2024

gavinchou mentioned this pull request Aug 19, 2024
Release Note 3.0.1 #39570
Closed

zddr mentioned this pull request Aug 19, 2024
[fix](mtmv)Mtmv support set both immediate and starttime #39573
Merged

morrySnow pushed a commit that referenced this pull request Aug 29, 2024
[fix](mtmv) Mtmv support set both immediate and starttime (#39573)
fb509ee
extends: #36805 Previously, if the user set immediate execution, the current time would replace the user's set start time

yiguolei added dev/2.1.6-merged and removed dev/2.1.x labels Sep 5, 2024

dataroaring pushed a commit that referenced this pull request Sep 26, 2024
[fix](mtmv) Mtmv support set both immediate and starttime (#39573)
7ef4c13
extends: #36805 Previously, if the user set immediate execution, the current time would replace the user's set start time

Uh oh!

Conversation

CalvinKirs commented Jun 25, 2024

Proposed changes

Uh oh!

doris-robot commented Jun 25, 2024

Uh oh!

CalvinKirs commented Jun 26, 2024

Uh oh!

doris-robot commented Jun 26, 2024

Uh oh!

doris-robot commented Jun 26, 2024

Uh oh!

doris-robot commented Jun 26, 2024

Uh oh!

zddr left a comment

Choose a reason for hiding this comment

Uh oh!

zddrJun 26, 2024

Choose a reason for hiding this comment

Uh oh!

CalvinKirsJun 27, 2024

Choose a reason for hiding this comment

Uh oh!

zddrJun 26, 2024

Choose a reason for hiding this comment

Uh oh!

CalvinKirsJun 27, 2024

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jun 27, 2024

Uh oh!

CalvinKirs commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

doris-robot commented Jun 27, 2024

Uh oh!

CalvinKirs commented Jul 1, 2024

Uh oh!

doris-robot commented Jul 1, 2024

Uh oh!

doris-robot commented Jul 1, 2024

Uh oh!

doris-robot commented Jul 1, 2024

Uh oh!

CalvinKirs commented Jul 2, 2024

Uh oh!

doris-robot commented Jul 2, 2024

Uh oh!

doris-robot commented Jul 2, 2024

Uh oh!

doris-robot commented Jul 2, 2024

Uh oh!

CalvinKirs commented Jul 3, 2024

Uh oh!

JNSimba left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jul 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants