Skip to content

Refactor alter job process - #1613

Closed
morningman wants to merge 41 commits into
apache:masterfrom
morningman:refactor_alter_job
Closed

Refactor alter job process#1613
morningman wants to merge 41 commits into
apache:masterfrom
morningman:refactor_alter_job

Conversation

@morningman

@morningmanmorningman commented Aug 8, 2019

Copy link
Copy Markdown
Contributor

ISSUE: #1429

kangkaisen
kangkaisen reviewed Aug 8, 2019
View reviewed changes
Comment threadfe/src/main/java/org/apache/doris/load/Load.java Outdated
Comment threadfe/src/main/java/org/apache/doris/planner/StreamLoadScanNode.java
Comment threadfe/src/main/java/org/apache/doris/alter/AlterJobV2.java
}

protected void runPendingJob() {
throw new NotImplementedException();

@kangkaisenkangkaisenAug 8, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making the AlterJobV2 to a abstract class is more reasonable？

@morningmanmorningmanAug 8, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I will change it after all test are done

@kangkaisenkangkaisenAug 9, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

Comment threadfe/src/main/java/org/apache/doris/alter/RollupJobV2.java Outdated
// send all tasks and wait them finished
AgentTaskQueue.addBatchTask(batchTask);
AgentTaskExecutor.submit(batchTask);
// max timeout is 30 seconds

@kangkaisenkangkaisenAug 8, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

30 seconds is certainly enough？ if the table has 1000 partitions and each partition has 100 buckets and only 10 BE.

@morningmanmorningmanAug 8, 2019

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I should change it to a configuration. And default value is 30 sec.
Because I don't want to stuck in here for a long time if error happens. But for
the extreme case like you said, user can always modify the config to satisfy their case.

@kangkaisenkangkaisenAug 9, 2019

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. configuration is OK, like create table logic.

Comment threadfe/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
Comment threadfe/src/main/java/org/apache/doris/task/AgentBatchTask.java
@morningman
morningmanforce-pushed the refactor_alter_job branch 3 times, most recently from ec102bd to 609b0faCompareAugust 13, 2019 09:17
@morningmanmorningman mentioned this pull request Aug 23, 2019
Merged
luwei16 added a commit to luwei16/Doris that referenced this pull request Apr 7, 2023
@luwei16
Force merge on write to false in cloud mode (apache#1613)
84d3a4e
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kangkaisenkangkaisenkangkaisen left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@morningman@kangkaisen