Skip to content

[RoutineLoad] Support modify routine load job - #4158

Merged
morningman merged 12 commits into
apache:masterfrom
morningman:modify_routine_load
Aug 6, 2020
Merged

[RoutineLoad] Support modify routine load job#4158
morningman merged 12 commits into
apache:masterfrom
morningman:modify_routine_load

Conversation

@morningman

Copy link
Copy Markdown
Contributor

Proposed changes

Support ALTER ROUTINE LOAD JOB stmt, for example:

alter routine load db1.label1
properties
(
"desired_concurrent_number"="3",
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200",
"strict_mode" = "false",
"timezone" = "+08:00"
)

Details can be found in alter-routine-load.md

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist

  • I have create an issue on fix[New Feature] Support modify routine load property #4157, and have described the bug/feature there in detail
  • Compiling and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • If this change need a document change, I have updated the document
  • Any dependent changes have been merged

@morningmanmorningman added kind/feature Categorizes issue or PR as related to a new feature. area/load Issues or PRs related to all kinds of load labels Jul 23, 2020
@morningmanmorningman self-assigned this Jul 23, 2020
RoutineLoadJob job = checkPrivAndGetJob(stmt.getDbName(), stmt.getLabel());
if (stmt.hasDataSourceProperty()
&& !stmt.getDataSourceProperties().getType().equalsIgnoreCase(job.dataSourceType.name())) {
throw new DdlException("The spciefied job type is not: " + stmt.getDataSourceProperties().getType());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

‘spciefied’ Wrong characters

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

{:
RESULT = new AlterDatabaseRename(dbName, newDbName);
:}
| KW_ALTER KW_ROUTINE KW_LOAD job_label:jobLabel opt_properties:properties

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| KW_ALTER KW_ROUTINE KW_LOAD job_label:jobLabel opt_properties:properties
| KW_ALTER KW_ROUTINE KW_LOAD job_label:jobLabel opt_properties:jobProperties

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

{:
RESULT = new AlterDatabaseRename(dbName, newDbName);
:}
| KW_ALTER KW_ROUTINE KW_LOAD job_label:jobLabel opt_properties:jobProperties

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe KW_ALTER KW_ROUTINE KW_LOAD KW_FOR job_label:jobLabel is better. The other routine load stmt also has KW_FOR.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

}
}

abstract public void modifyProperties(AlterRoutineLoadStmt stmt) throws DdlException;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The job properties maybe could be modified in here instead of the subclass KafkaRoutineLoad.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could do this when we got second derived class from RoutineLoadJob


abstract public void modifyProperties(AlterRoutineLoadStmt stmt) throws DdlException;

abstract public void replayModifyProperties(AlterRoutineLoadJobOperationLog log);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

Comment threaddocs/.vuepress/sidebar/zh-CN.js Outdated
title: "DML",
directoryPath: "Data Manipulation/",
children: [
"alter-routine-load",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put it between pause routine load and stop routine load.

@EmmyMiao87EmmyMiao87 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@EmmyMiao87EmmyMiao87 added the approved Indicates a PR has been approved by one committer. label Aug 6, 2020
@morningman
morningman merged commit 237c080 into apache:masterAug 6, 2020
@EmmyMiao87EmmyMiao87 mentioned this pull request Aug 17, 2020
bobhan1 pushed a commit to bobhan1/doris that referenced this pull request Jul 15, 2025
csun5285 pushed a commit to csun5285/doris that referenced this pull request Sep 23, 2025
…pache#4158)
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.area/loadIssues or PRs related to all kinds of loadkind/featureCategorizes issue or PR as related to a new feature.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[New Feature] Support modify routine load property

4 participants

@morningman@wutiangan@EmmyMiao87@morningman-cmy