[bugfix](iceberg)Read error when timestamp does not have time zone by wuwenchi · Pull Request #36141 · apache/doris

wuwenchi · 2024-06-11T13:12:16Z

Proposed changes

Iceberg has two types of timestamps, one with time zone and one without time zone. When dealing with timestamps that do not match the time zone, the time zone offset cannot be added to the timestamp value.

doris-robot · 2024-06-11T13:12:22Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

wuwenchi · 2024-06-11T13:16:34Z

run buildall

doris-robot · 2024-06-11T14:01:29Z

TPC-H: Total hot run time: 39591 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9b71f59dc498bc1d49877df39b4e921a9410981a, data reload: false
------ Round 1 ----------------------------------
q1	17632	4315	4296	4296
q2	2018	193	201	193
q3	10430	1167	1070	1070
q4	10202	760	761	760
q5	7492	2635	2635	2635
q6	215	134	132	132
q7	944	621	609	609
q8	9211	2044	2067	2044
q9	9060	6463	6451	6451
q10	8984	3763	3688	3688
q11	451	239	228	228
q12	476	231	228	228
q13	17761	3030	2989	2989
q14	264	211	209	209
q15	524	482	472	472
q16	511	373	377	373
q17	955	676	659	659
q18	7857	7418	7245	7245
q19	7739	1486	1522	1486
q20	652	316	312	312
q21	4929	3205	3195	3195
q22	383	329	317	317
Total cold run time: 118690 ms
Total hot run time: 39591 ms
----- Round 2, with runtime_filter_mode=off -----
q1	4390	4284	4227	4227
q2	339	260	261	260
q3	2985	2983	2930	2930
q4	1943	1727	1739	1727
q5	5540	5495	5474	5474
q6	218	128	139	128
q7	2192	1856	1838	1838
q8	3272	3394	3422	3394
q9	8710	8663	8687	8663
q10	4119	3833	3697	3697
q11	584	489	508	489
q12	845	635	653	635
q13	16225	3158	3205	3158
q14	310	273	295	273
q15	524	495	481	481
q16	504	437	429	429
q17	1796	1503	1477	1477
q18	8250	7889	7700	7700
q19	1753	1632	1529	1529
q20	2115	1874	1872	1872
q21	5204	4968	4830	4830
q22	614	574	567	567
Total cold run time: 72432 ms
Total hot run time: 55778 ms

wuwenchi · 2024-06-11T14:09:05Z

run feut

doris-robot · 2024-06-11T14:12:25Z

TPC-DS: Total hot run time: 170105 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9b71f59dc498bc1d49877df39b4e921a9410981a, data reload: false
query1	931	383	372	372
query2	6444	2424	2242	2242
query3	6627	209	211	209
query4	18409	17324	17557	17324
query5	3572	448	445	445
query6	242	156	162	156
query7	4582	302	286	286
query8	330	287	306	287
query9	8536	2343	2325	2325
query10	553	311	290	290
query11	10661	10103	9907	9907
query12	125	91	86	86
query13	1643	379	360	360
query14	9986	7702	7025	7025
query15	262	200	193	193
query16	8057	322	287	287
query17	1812	565	538	538
query18	2154	287	296	287
query19	232	162	162	162
query20	108	85	83	83
query21	210	135	131	131
query22	4981	4356	4474	4356
query23	34126	33686	33479	33479
query24	10945	2860	2851	2851
query25	587	347	351	347
query26	1031	151	147	147
query27	2246	315	315	315
query28	6772	2034	2007	2007
query29	831	610	582	582
query30	240	147	149	147
query31	914	738	729	729
query32	89	52	57	52
query33	733	295	268	268
query34	888	455	468	455
query35	715	656	608	608
query36	1095	936	930	930
query37	134	71	65	65
query38	2866	2731	2734	2731
query39	835	788	800	788
query40	204	125	122	122
query41	52	50	53	50
query42	114	95	95	95
query43	590	553	539	539
query44	1236	729	719	719
query45	197	169	162	162
query46	1096	745	754	745
query47	1811	1790	1774	1774
query48	369	292	288	288
query49	858	434	388	388
query50	771	384	378	378
query51	6844	6708	6631	6631
query52	98	93	87	87
query53	354	289	288	288
query54	876	433	432	432
query55	75	72	74	72
query56	267	253	254	253
query57	1128	1063	1039	1039
query58	230	255	240	240
query59	3449	3319	2998	2998
query60	284	261	297	261
query61	91	87	116	87
query62	596	480	435	435
query63	314	292	289	289
query64	8687	2195	1658	1658
query65	3163	3075	3127	3075
query66	733	330	340	330
query67	15183	15002	14794	14794
query68	6220	533	542	533
query69	675	460	413	413
query70	1148	1130	1103	1103
query71	479	277	266	266
query72	7175	5257	2718	2718
query73	772	322	316	316
query74	5832	5451	5477	5451
query75	4032	2699	2659	2659
query76	3962	919	927	919
query77	667	303	301	301
query78	10281	9790	9765	9765
query79	3228	511	513	511
query80	1097	450	455	450
query81	547	217	217	217
query82	280	99	97	97
query83	189	164	162	162
query84	272	90	84	84
query85	930	268	324	268
query86	367	328	306	306
query87	3315	3084	3082	3082
query88	4064	2474	2436	2436
query89	477	378	378	378
query90	2034	180	184	180
query91	121	98	97	97
query92	56	49	47	47
query93	4164	509	499	499
query94	1284	183	178	178
query95	391	305	322	305
query96	591	272	262	262
query97	3219	3014	2999	2999
query98	210	194	188	188
query99	1263	857	834	834
Total cold run time: 274516 ms
Total hot run time: 170105 ms

doris-robot · 2024-06-11T14:17:57Z

ClickBench: Total hot run time: 31.29 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9b71f59dc498bc1d49877df39b4e921a9410981a, data reload: false
query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.10	0.11
query5	0.51	0.49	0.49
query6	1.13	0.72	0.73
query7	0.02	0.01	0.02
query8	0.06	0.04	0.05
query9	0.54	0.49	0.50
query10	0.55	0.56	0.54
query11	0.16	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.60
query14	0.76	0.77	0.80
query15	0.83	0.81	0.83
query16	0.35	0.38	0.37
query17	1.01	1.01	0.96
query18	0.24	0.22	0.26
query19	1.76	1.67	1.67
query20	0.02	0.00	0.01
query21	15.42	0.65	0.65
query22	3.63	7.60	2.40
query23	18.30	1.48	1.30
query24	2.09	0.24	0.21
query25	0.15	0.09	0.09
query26	0.27	0.18	0.18
query27	0.08	0.08	0.08
query28	13.29	1.02	1.01
query29	12.68	3.29	3.26
query30	0.26	0.06	0.06
query31	2.85	0.40	0.38
query32	3.26	0.46	0.47
query33	2.94	2.89	2.93
query34	16.94	4.66	4.62
query35	4.73	4.60	4.50
query36	0.64	0.48	0.49
query37	0.17	0.16	0.15
query38	0.16	0.14	0.14
query39	0.05	0.03	0.04
query40	0.17	0.14	0.15
query41	0.08	0.04	0.04
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 108.96 s
Total hot run time: 31.29 s

morningman

LGTM

github-actions · 2024-06-17T10:41:56Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-06-17T10:41:58Z

PR approved by anyone and no changes requested.

…r 2.1 (#36435) bp: #36141

…36141) ## Proposed changes Iceberg has two types of timestamps, one with time zone and one without time zone. When dealing with timestamps that do not match the time zone, the time zone offset cannot be added to the timestamp value.

…pache#36141) ## Proposed changes Iceberg has two types of timestamps, one with time zone and one without time zone. When dealing with timestamps that do not match the time zone, the time zone offset cannot be added to the timestamp value.

wuwenchi added 2 commits June 11, 2024 14:32

fix
6752365

add test
9b71f59

morningman approved these changes Jun 17, 2024
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Jun 17, 2024

github-actionsBot added the reviewed label Jun 17, 2024

zddr approved these changes Jun 17, 2024
View reviewed changes

morningman added the dev/2.1.x label Jun 17, 2024

morningman merged commit 4517faf into apache:masterJun 17, 2024

wuwenchi mentioned this pull request Jun 18, 2024
[bugfix](iceberg)Read error when timestamp does not have time zone for 2.1 #36435
Merged

morningman pushed a commit that referenced this pull request Jun 20, 2024
[bugfix](iceberg)Read error when timestamp does not have time zone fo…
ac0f6e7
…r 2.1 (#36435) bp: #36141

morningman added dev/2.1.4-merged and removed dev/2.1.x labels Jun 20, 2024

dataroaring added the dev/3.0.0-merged label Aug 5, 2024

Navigation Menu

Uh oh!

[bugfix](iceberg)Read error when timestamp does not have time zone - #36141

[bugfix](iceberg)Read error when timestamp does not have time zone#36141
morningman merged 2 commits into
apache:masterfrom
wuwenchi:fix-iceberg-ts-ntz

wuwenchi commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

wuwenchi commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

wuwenchi commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

morningman left a comment

Uh oh!

github-actionsBot commented Jun 17, 2024

Uh oh!

github-actionsBot commented Jun 17, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

wuwenchi commented Jun 11, 2024

Proposed changes

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

wuwenchi commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

wuwenchi commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

doris-robot commented Jun 11, 2024

Uh oh!

morningman left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Jun 17, 2024

Uh oh!

github-actionsBot commented Jun 17, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants