Skip to content

[deps](jindofs) update jindofs from 6.3.4 to 6.8.2 - #49886

Merged
morningman merged 1 commit into
apache:masterfrom
morningman:upgrade_jindofs
Apr 9, 2025
Merged

[deps](jindofs) update jindofs from 6.3.4 to 6.8.2#49886
morningman merged 1 commit into
apache:masterfrom
morningman:upgrade_jindofs

Conversation

@morningman

@morningmanmorningman commented Apr 8, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Problem Summary:

There are some bugs in 6.3.4 that will cause 403 error when accessing oss-hdfs

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Apr 8, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman

morningman commented Apr 8, 2025

Copy link
Copy Markdown
ContributorAuthor

run buildall

@doris-robot

doris-robot commented Apr 8, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 35086 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 01acae44565cc81afa3588a97e07f2908bba67ef, data reload: false
------ Round 1 ----------------------------------
q1	26030	5069	5110	5069
q2	2073	284	193	193
q3	10378	1244	678	678
q4	10229	1013	536	536
q5	7522	2391	2340	2340
q6	191	163	133	133
q7	913	734	610	610
q8	9321	1292	1087	1087
q9	6958	5159	5129	5129
q10	6827	2301	1917	1917
q11	478	283	265	265
q12	351	359	219	219
q13	17754	3669	3131	3131
q14	232	221	208	208
q15	527	503	480	480
q16	620	623	572	572
q17	596	857	383	383
q18	7615	7273	7197	7197
q19	1227	962	568	568
q20	354	335	229	229
q21	4515	3514	3173	3173
q22	1033	1007	969	969
Total cold run time: 115744 ms
Total hot run time: 35086 ms
----- Round 2, with runtime_filter_mode=off -----
q1	5137	5094	5100	5094
q2	243	329	232	232
q3	2128	2658	2310	2310
q4	1401	1843	1474	1474
q5	4550	4477	4359	4359
q6	213	167	125	125
q7	1982	1929	1741	1741
q8	2592	2526	2491	2491
q9	7176	7159	7099	7099
q10	2992	3180	2730	2730
q11	598	503	494	494
q12	667	756	599	599
q13	3496	3836	3283	3283
q14	277	302	275	275
q15	578	471	457	457
q16	680	714	655	655
q17	1152	1568	1349	1349
q18	7740	7567	7391	7391
q19	811	781	931	781
q20	2014	1990	1804	1804
q21	5181	4719	4608	4608
q22	1077	1032	979	979
Total cold run time: 52685 ms
Total hot run time: 50330 ms

@doris-robot

doris-robot commented Apr 8, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 186490 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 01acae44565cc81afa3588a97e07f2908bba67ef, data reload: false
query1	1008	483	463	463
query2	6570	1960	1934	1934
query3	6742	228	218	218
query4	25813	23203	23366	23203
query5	4358	600	447	447
query6	290	204	185	185
query7	4626	491	280	280
query8	309	247	238	238
query9	8605	2595	2580	2580
query10	477	309	263	263
query11	15225	14990	14828	14828
query12	163	109	105	105
query13	1637	523	404	404
query14	8722	5994	6157	5994
query15	193	188	167	167
query16	7114	640	462	462
query17	908	699	614	614
query18	1972	413	326	326
query19	193	181	161	161
query20	123	124	119	119
query21	219	127	105	105
query22	4159	4171	4177	4171
query23	33953	32966	32968	32966
query24	8507	2408	2414	2408
query25	543	499	425	425
query26	1250	273	162	162
query27	2741	506	337	337
query28	4416	2414	2404	2404
query29	785	588	447	447
query30	286	224	199	199
query31	931	869	769	769
query32	77	69	63	63
query33	586	371	387	371
query34	784	849	493	493
query35	770	822	752	752
query36	952	993	905	905
query37	114	97	72	72
query38	4147	4086	4080	4080
query39	1453	1408	1394	1394
query40	211	118	105	105
query41	55	55	52	52
query42	121	105	105	105
query43	482	508	472	472
query44	1357	789	792	789
query45	175	168	168	168
query46	825	1014	607	607
query47	1765	1810	1773	1773
query48	373	409	307	307
query49	782	517	426	426
query50	632	659	389	389
query51	4164	4164	4081	4081
query52	123	105	95	95
query53	222	247	177	177
query54	570	575	505	505
query55	91	83	85	83
query56	301	294	279	279
query57	1139	1147	1066	1066
query58	261	252	253	252
query59	2683	2816	2614	2614
query60	320	317	297	297
query61	141	130	127	127
query62	779	733	634	634
query63	217	182	188	182
query64	4324	1014	790	790
query65	4307	4235	4255	4235
query66	1171	416	314	314
query67	15882	15672	15379	15379
query68	8659	879	505	505
query69	471	295	264	264
query70	1182	1140	1108	1108
query71	437	309	299	299
query72	5643	4769	4758	4758
query73	747	626	344	344
query74	8930	8934	8680	8680
query75	4011	3202	2677	2677
query76	3620	1182	747	747
query77	792	380	282	282
query78	10047	10207	9316	9316
query79	2198	808	565	565
query80	609	513	439	439
query81	468	259	225	225
query82	438	127	98	98
query83	285	248	247	247
query84	297	112	81	81
query85	782	341	377	341
query86	337	312	293	293
query87	4473	4471	4533	4471
query88	3122	2219	2197	2197
query89	386	317	280	280
query90	1932	208	208	208
query91	146	142	113	113
query92	84	63	61	61
query93	1130	924	574	574
query94	672	421	305	305
query95	381	294	278	278
query96	486	552	279	279
query97	3165	3297	3163	3163
query98	229	213	204	204
query99	1425	1395	1271	1271
Total cold run time: 272778 ms
Total hot run time: 186490 ms

@doris-robot

doris-robot commented Apr 8, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.4 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 01acae44565cc81afa3588a97e07f2908bba67ef, data reload: false
query1	0.04	0.04	0.03
query2	0.13	0.10	0.11
query3	0.24	0.18	0.19
query4	1.60	0.20	0.19
query5	0.59	0.57	0.59
query6	1.18	0.71	0.71
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.57	0.53	0.52
query10	0.59	0.58	0.59
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.62	0.60	0.59
query14	2.81	2.74	2.67
query15	0.94	0.85	0.83
query16	0.39	0.39	0.38
query17	1.01	1.09	1.04
query18	0.22	0.20	0.20
query19	1.93	1.93	1.87
query20	0.01	0.01	0.01
query21	15.35	0.89	0.54
query22	0.76	1.34	0.89
query23	14.70	1.41	0.64
query24	6.99	1.37	0.93
query25	0.51	0.13	0.09
query26	0.68	0.18	0.14
query27	0.05	0.05	0.05
query28	9.70	0.84	0.42
query29	12.59	3.90	3.29
query30	0.26	0.09	0.06
query31	2.83	0.58	0.38
query32	3.22	0.55	0.47
query33	3.05	3.03	3.11
query34	15.77	5.11	4.46
query35	4.50	4.47	4.50
query36	0.65	0.49	0.49
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.18	0.13	0.13
query41	0.09	0.02	0.03
query42	0.04	0.02	0.02
query43	0.03	0.04	0.02
Total cold run time: 105.34 s
Total hot run time: 31.4 s

@doris-robot

doris-robot commented Apr 8, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

CategoryCoverage
Function Coverage52.26% (14032/26848)
Line Coverage41.03% (121045/294998)
Region Coverage39.79% (61629/154898)
Branch Coverage34.46% (30855/89542)

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Apr 9, 2025
@github-actions

github-actionsBot commented Apr 9, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Apr 9, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@morningman
morningman merged commit e11d62a into apache:masterApr 9, 2025
This was referenced Apr 9, 2025
Merged
Merged
hello-stephen pushed a commit that referenced this pull request Apr 9, 2025
@morningman
[deps](jindofs) build.sh support any version of jindofs (#49903)
ff7655b
### What problem does this PR solve?
Followup #49886 Use regexp to match the different version of jindofs jars.
So that we don't need to modify `build.sh` again once the version of
jindofs is changed
yiguolei pushed a commit that referenced this pull request Apr 10, 2025
@morningman
[deps](jindofs) add jindofs for 2.1 (#49259)(#49371)(#49374)(#49502)(#…
826cbfd
…49503)(#49886)(#49903) (#49908)
bp (#49259)(#49371)(#49374)(#49502)(#49503)(#49886)(#49903)
morningman added a commit to morningman/doris that referenced this pull request Apr 21, 2025
@morningman
[deps](jindofs) build.sh support any version of jindofs (apache#49903)
6047794
Followup apache#49886
Use regexp to match the different version of jindofs jars.
So that we don't need to modify `build.sh` again once the version of
jindofs is changed
morningman added a commit to morningman/doris that referenced this pull request Apr 22, 2025
@morningman
[deps](jindofs) add jindofs for 2.1 (apache#49259)(apache#49371)(apac…
79bb9b1
…he#49374)(apache#49502)(apache#49503)(apache#49886)(apache#49903) (apache#49908)
bp (apache#49259)(apache#49371)(apache#49374)(apache#49502)(apache#49503)(apache#49886)(apache#49903)
morningman added a commit to morningman/doris that referenced this pull request Apr 22, 2025
@morningman
[deps](jindofs) build.sh support any version of jindofs (apache#49903)
b337e98
### What problem does this PR solve?
Followup apache#49886 Use regexp to match the different version of jindofs jars.
So that we don't need to modify `build.sh` again once the version of
jindofs is changed
@morningmanmorningman mentioned this pull request Apr 22, 2025
Merged
dataroaring pushed a commit that referenced this pull request Apr 23, 2025
@morningman@hello-stephen
[deps](jindofs) add jindofs for 2.1 (#49259)(#49371)(#49374)(#49502)(#…
8aff230
…49503)(#49886)(#49903)(#49963) (#50283)
bp (#49259)(#49371)(#49374)(#49502)(#49503)(#49886)(#49903)(#49963)
---------
Co-authored-by: Dongyang Li <lidongyang@selectdb.com>
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@morningmankoarz
[deps](jindofs) update jindofs from 6.3.4 to 6.8.2 (apache#49886)
c4b8575
### What problem does this PR solve?
Problem Summary:
There are some bugs in 6.3.4 that will cause 403 error when accessing
oss-hdfs
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@morningmankoarz
[deps](jindofs) build.sh support any version of jindofs (apache#49903)
9c48d71
### What problem does this PR solve?
Followup apache#49886 Use regexp to match the different version of jindofs jars.
So that we don't need to modify `build.sh` again once the version of
jindofs is changed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CalvinKirsCalvinKirsCalvinKirs approved these changes
@wuwenchiwuwenchiwuwenchi approved these changes
@hubgeterhubgeterhubgeter approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.10-mergeddev/3.0.6-mergedreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@morningman@hello-stephen@doris-robot@CalvinKirs@wuwenchi@hubgeter@dataroaring