[feature-wip](arrow-flight)(step3) Support authentication and user session by xinyiZzz · Pull Request #24772 · apache/doris

xinyiZzz · 2023-09-22T03:12:10Z

Proposed changes

Design Documentation Linked to #25514

Associated:
apache/arrow#37824
apache/arrow-adbc#1079

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

github-actions

clang-tidy made some suggestions

github-actions

clang-tidy made some suggestions

xinyiZzz · 2023-09-25T10:35:26Z

run buildall

doris-robot · 2023-09-25T11:04:26Z

TeamCity be ut coverage result:
Function Coverage: 36.31% (8111/22336)
Line Coverage: 28.44% (64874/228148)
Region Coverage: 27.32% (33643/123127)
Branch Coverage: 23.97% (17170/71620)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7703d02d404be6559fd5c9fb21fa7aa21afd87fd_7703d02d404be6559fd5c9fb21fa7aa21afd87fd/report/index.html

github-actions

clang-tidy made some suggestions

xinyiZzz · 2023-09-26T12:57:45Z

run buildall

doris-robot · 2023-09-26T13:34:52Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.01 seconds
stream load tsv: 591 seconds loaded 74807831229 Bytes, about 120 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17162108737 Bytes

doris-robot · 2023-09-26T13:47:37Z

TeamCity be ut coverage result:
Function Coverage: 36.34% (8144/22413)
Line Coverage: 28.44% (65165/229096)
Region Coverage: 27.40% (33784/123301)
Branch Coverage: 24.07% (17250/71670)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8d64d724f309c6e073a37bda190039287e619eab_8d64d724f309c6e073a37bda190039287e619eab/report/index.html

xinyiZzz · 2023-09-26T15:27:28Z

run buildall

doris-robot · 2023-09-26T15:53:16Z

TeamCity be ut coverage result:
Function Coverage: 36.33% (8142/22413)
Line Coverage: 28.44% (65154/229093)
Region Coverage: 27.39% (33776/123293)
Branch Coverage: 24.06% (17241/71664)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e231c8a9b1d1a8ec179c9ad5da74595382a9ce18_e231c8a9b1d1a8ec179c9ad5da74595382a9ce18/report/index.html

doris-robot · 2023-09-26T16:06:44Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.88 seconds
stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162100595 Bytes

xinyiZzz · 2023-09-27T01:43:47Z

run buildall

doris-robot · 2023-09-27T02:08:06Z

TeamCity be ut coverage result:
Function Coverage: 36.33% (8142/22413)
Line Coverage: 28.44% (65163/229093)
Region Coverage: 27.41% (33790/123295)
Branch Coverage: 24.06% (17246/71666)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c5039bbfd6b3a67e0dc8e84be0be4048d72e1221_c5039bbfd6b3a67e0dc8e84be0be4048d72e1221/report/index.html

doris-robot · 2023-09-27T02:25:19Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.59 seconds
stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162225259 Bytes

xinyiZzz · 2023-09-27T05:03:22Z

run buildall

doris-robot · 2023-09-27T05:31:52Z

TeamCity be ut coverage result:
Function Coverage: 36.32% (8140/22413)
Line Coverage: 28.43% (65135/229103)
Region Coverage: 27.39% (33774/123301)
Branch Coverage: 24.05% (17238/71670)
Coverage Report: http://coverage.selectdb-in.cc/coverage/566c100fb1b53887c1f5e0b9d70ee1622488e8f4_566c100fb1b53887c1f5e0b9d70ee1622488e8f4/report/index.html

doris-robot · 2023-09-27T05:34:32Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.44 seconds
stream load tsv: 574 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 64 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17162543778 Bytes

yiguolei · 2023-09-27T06:35:09Z

+ @ConfField(description = {"Arrow Flight Server中用户token的存活时间，自上次写入后过期时间，单位分钟，默认值为4320，即3天",
+ "The alive time of the user token in Arrow Flight Server, expire after write, unit minutes,"
+ + "the default value is 4320, which is 3 days"})
+ public static int arrow_flight_token_alive_time = 4320;


change to arrow_flight_token_alive_time_secs

yiguolei

LGTM

github-actions · 2023-09-27T06:38:25Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-09-27T06:38:27Z

PR approved by anyone and no changes requested.

HappenLee

LGTM

…ssion (apache#24772)

xinyiZzz marked this pull request as draft September 22, 2023 03:12

github-actionsBot reviewed Sep 22, 2023
View reviewed changes

Comment threadbe/src/service/arrow_flight/flight_sql_server_auth_handler.cpp Outdated
Comment threadbe/src/service/arrow_flight/flight_sql_server_auth_handler.h Outdated

github-actionsBot reviewed Sep 22, 2023
View reviewed changes

xinyiZzz force-pushed the 20230922_support_arrow_flight_sql7 branch from fc38284 to f15f6b1Compare September 25, 2023 09:10

xinyiZzz marked this pull request as ready for review September 25, 2023 10:19

github-actionsBot reviewed Sep 26, 2023
View reviewed changes

Comment threadbe/src/service/arrow_flight/auth_server_middleware.cpp

xinyiZzz force-pushed the 20230922_support_arrow_flight_sql7 branch 2 times, most recently from 4179ebb to 8d64d72Compare September 26, 2023 12:56

xinyiZzz force-pushed the 20230922_support_arrow_flight_sql7 branch from 8d64d72 to e231c8aCompare September 26, 2023 15:27

xinyiZzz added 6 commits September 27, 2023 09:42

1
1937b1f

3
d1058e1

4
b05716b

5
14ae665

6
c0ae5b0

7
c5039bb

xinyiZzz force-pushed the 20230922_support_arrow_flight_sql7 branch from e231c8a to c5039bbCompare September 27, 2023 01:43

Merge branch 'master' into 20230922_support_arrow_flight_sql7
566c100

yiguolei reviewed Sep 27, 2023
View reviewed changes

yiguolei approved these changes Sep 27, 2023
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Sep 27, 2023

github-actionsBot added the reviewed label Sep 27, 2023

HappenLee approved these changes Sep 27, 2023
View reviewed changes

yiguolei merged commit 87a30dc into apache:masterSep 27, 2023

vinlee19 pushed a commit to vinlee19/doris that referenced this pull request Oct 7, 2023
[feature-wip](arrow-flight)(step3) Support authentication and user se…
3122b87
…ssion (apache#24772)

xinyiZzz mentioned this pull request Oct 17, 2023
[Feature] Doris support Arrow Flight SQL protocol #25514
Closed
1 task

Uh oh!

Conversation

xinyiZzz commented Sep 22, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xinyiZzz commented Sep 25, 2023

Uh oh!

doris-robot commented Sep 25, 2023

Uh oh!

github-actionsBot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

xinyiZzz commented Sep 26, 2023

Uh oh!

doris-robot commented Sep 26, 2023

Uh oh!

doris-robot commented Sep 26, 2023

Uh oh!

xinyiZzz commented Sep 26, 2023

Uh oh!

doris-robot commented Sep 26, 2023

Uh oh!

doris-robot commented Sep 26, 2023

Uh oh!

xinyiZzz commented Sep 27, 2023

Uh oh!

doris-robot commented Sep 27, 2023

Uh oh!

doris-robot commented Sep 27, 2023

Uh oh!

xinyiZzz commented Sep 27, 2023

Uh oh!

doris-robot commented Sep 27, 2023

Uh oh!

doris-robot commented Sep 27, 2023

Uh oh!

yiguoleiSep 27, 2023

Choose a reason for hiding this comment

Uh oh!

yiguolei left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Sep 27, 2023

Uh oh!

github-actionsBot commented Sep 27, 2023

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xinyiZzz commented Sep 22, 2023•
edited
Loading