Skip to content

[Feature] ADD: show create routine load - #6067

Closed
stalary wants to merge 21 commits into
apache:masterfrom
stalary:show_create_routine_load
Closed

[Feature] ADD: show create routine load#6067
stalary wants to merge 21 commits into
apache:masterfrom
stalary:show_create_routine_load

Conversation

@stalary

@stalarystalary commented Jun 21, 2021•
edited by EmmyMiao87
Loading

Copy link
Copy Markdown
Contributor

Proposed changes

Add show create routine load

Fixed#5645

@stalary

stalary commented Jun 21, 2021

Copy link
Copy Markdown
ContributorAuthor

#5645

@stalarystalary changed the title [WIP] ADD: show create routine load[Feature] ADD: show create routine loadJun 21, 2021
@EmmyMiao87

EmmyMiao87 commented Jun 23, 2021

Copy link
Copy Markdown
Contributor

In fact, these information will be displayed in the results of show routine load.
Under what circumstances will a show create routine load syntax be needed?

@EmmyMiao87EmmyMiao87 self-assigned this Jun 23, 2021
@morningman

morningman commented Jun 23, 2021

Copy link
Copy Markdown
Contributor

In fact, these information will be displayed in the results of show routine load.
Under what circumstances will a show create routine load syntax be needed?

Just like show create table, user may may need to create routine load job again.

@stalary

stalary commented Jun 23, 2021

Copy link
Copy Markdown
ContributorAuthor

For example, I stopped it and then tried to restart it

EmmyMiao87
EmmyMiao87 reviewed Jun 23, 2021
View reviewed changes
Comment threadfe/fe-core/src/main/java/org/apache/doris/analysis/ShowCreateRoutineLoadStmt.java Outdated
Comment threadfe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java Outdated
Comment threadfe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java Outdated
@EmmyMiao87

EmmyMiao87 commented Jun 23, 2021

Copy link
Copy Markdown
Contributor

You need to consider a question.
Whether the routine load after stopping also needs to be displayed.

The routine load after stopping may have the same name.
So if show, it may produce multiple results.
The one way is that using 'show create routine load' to display running routine load, using 'show all create routine load' to displays all routine loads with the same name include stopped.

@EmmyMiao87EmmyMiao87 added area/routine load kind/feature Categorizes issue or PR as related to a new feature. labels Jun 23, 2021
@stalary

stalary commented Jun 23, 2021

Copy link
Copy Markdown
ContributorAuthor

You need to consider a question.
Whether the routine load after stopping also needs to be displayed.

The routine load after stopping may have the same name.
So if show, it may produce multiple results.
The one way is that using 'show create routine load' to display running routine load, using 'show all create routine load' to displays all routine loads with the same name include stopped.

Well, I've decided to support both

@EmmyMiao87

EmmyMiao87 commented Jun 23, 2021

Copy link
Copy Markdown
Contributor

At the moment my requirement is just to show running, do we need to support all of them？Show all create routine load is not very useful to me right now.

This is no problem.
It needs to be stated on the document that only the routine load of running or paused is displayed.

@stalary

stalary commented Jun 25, 2021

Copy link
Copy Markdown
ContributorAuthor

PTAL @EmmyMiao87

EmmyMiao87
EmmyMiao87 reviewed Jun 28, 2021
View reviewed changes
Comment threaddocs/zh-CN/sql-reference/sql-statements/Data Manipulation/SHOW CREATE ROUTINE LOAD.md Outdated

# SHOW CREATE ROUTINE LOAD
## description
该语句用于展示用户自定义例行导入作业的创建语句

@EmmyMiao87EmmyMiao87Jun 28, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
该语句用于展示用户自定义例行导入作业的创建语句
该语句用于展示例行导入作业的创建语句

Comment threadfe/fe-core/src/main/java/org/apache/doris/load/routineload/KafkaRoutineLoadJob.java Outdated
Comment threadfe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
Comment threadfe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java Outdated
dbName,
tableName,
PrivPredicate.LOAD)) {
ErrorReport.reportAnalysisException(ErrorCode.ERR_TABLEACCESS_DENIED_ERROR, "LOAD",

@EmmyMiao87EmmyMiao87Jun 28, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If user don’t have permission, we can simply not display it without reporting an error

Comment threadfe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java Outdated
@EmmyMiao87

EmmyMiao87 commented Jun 28, 2021•
edited
Loading

Copy link
Copy Markdown
Contributor
  1. Please add some UT~
  2. Maybe you need to rebase master so that P0(NewDorisTest) could be passed.

morningmanand others added 7 commits June 28, 2021 22:43
@morningman@stalary
[Enhance] Support show unrecoverable tablets (apache#6045)
9ae16e1
* [Enhance] Support show unrecoverable tablets
The unrecoverable tablets are tablets which non of their replicas are healthy.
We should be able to find out these tablets then manual intervention.
And these tablets should not be added to the tablet scheduler.
@e0c9@stalary
[Feature] Support recording custom number of backup and restore task …
b2a7794
…information (apache#5947)
* Record all backup jobs and support where clause
Xinyi Zouzouxinyi@stalary
fix spring boot web maximum upload file limit config (apache#6070)
7aa034d
Co-authored-by: zouxinyi <zouxinyi@baidu.com>
@morningman@stalary
[Build][ARM] Fix some compilation problems on ARM64 (apache#6076)
b07ab37
1. Disable libhdfs3 on ARM, because it doesn't support ARM now.
2. Add compilation doc for ARM64
@morningman@stalary
[Bug] Fix bug that select into outfile in parquet format may cause NPE (
f7ba232
apache#6054)
1. check the parquet schema property on FE side.
2. auto generate parquet schema if not specified.
@EmmyMiao87@stalary
[Docs] Fix README.md (apache#6084)
851b771
@e0c9@stalary
fix (apache#6081)
e970572
@stalarystalary closed this Jun 28, 2021
@stalary
stalary deleted the show_create_routine_load branch June 28, 2021 14:58
@stalarystalary mentioned this pull request Jul 4, 2021
Merged
10 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@EmmyMiao87EmmyMiao87EmmyMiao87 left review comments

Assignees

@EmmyMiao87EmmyMiao87

Labels

area/routine loadkind/featureCategorizes issue or PR as related to a new feature.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add show create routine load support

6 participants

@stalary@EmmyMiao87@morningman@e0c9@HappenLee@stdpain