Skip to content

Support check committed txns before catalog drop meta, like db, table, partition etc - #4029

Merged
chaoyli merged 10 commits into
apache:masterfrom
caiconghui:drop_check
Jul 28, 2020
Merged

Support check committed txns before catalog drop meta, like db, table, partition etc#4029
chaoyli merged 10 commits into
apache:masterfrom
caiconghui:drop_check

Conversation

@caiconghui

@caiconghuicaiconghui commented Jul 6, 2020•
edited
Loading

Copy link
Copy Markdown
Contributor

This PR is to ensure that dropped db , table or partition can be with normal state after recovered by user. Commited txns can not be aborted, because the partitions's commited versions have been changed, and some tablets may already have new visible versions. If user just don't want the meta(db, table or partition) anymore, just use dropp instead of drop to skip committed txn check.

@caiconghui

caiconghui commented Jul 6, 2020

Copy link
Copy Markdown
ContributorAuthor

for #3988

morningman
morningman reviewed Jul 6, 2020
View reviewed changes
Comment threadfe/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java
Comment threadfe/src/main/java/org/apache/doris/catalog/Catalog.java Outdated
Comment threadfe/src/main/java/org/apache/doris/analysis/DropTableStmt.java Outdated
Comment threadfe/src/main/java/org/apache/doris/analysis/DropTableStmt.java Outdated
Comment threadfe/src/main/java/org/apache/doris/analysis/DropTableStmt.java Outdated
Comment threadfe/src/main/java/org/apache/doris/analysis/DropTableStmt.java Outdated
Comment threadfe/src/main/java/org/apache/doris/catalog/Catalog.java Outdated
@morningman

morningman commented Jul 6, 2020

Copy link
Copy Markdown
Contributor

If I use DROPP Table to forcibly drop a table with COMMITTED txns.
Can those COMMITTED txns be cleaned automatically?

@caiconghui

caiconghui commented Jul 6, 2020•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

If I use DROPP Table to forcibly drop a table with COMMITTED txns.
Can those COMMITTED txns be cleaned automatically?

@morningman in fe, the txn will be publish timeout, and publish version daemon will try to finish transaction, it find that table have been dropped and will remove table committed info from txn, the txn will be visible. In be, it seems that txns wiil be cleaned by reporter handler?

@caiconghuicaiconghui changed the title Support check commited txns before catalog drop meta, like db, table, partition etcSupport check committed txns before catalog drop meta, like db, table, partition etcJul 6, 2020
@chaoyli

chaoyli commented Jul 19, 2020

Copy link
Copy Markdown
Contributor

I found if perform DROP database force, it's committed version will be recovered by RECOVER command?

@caiconghui

caiconghui commented Jul 19, 2020

Copy link
Copy Markdown
ContributorAuthor

I found if perform DROP database force, it's committed version will be recovered by RECOVER command?

@chaoyli I will fix it, make force drop operation do not recycle meta.

chaoyli
chaoyli previously approved these changes Jul 27, 2020
View reviewed changes
Comment threadfe/src/main/cup/sql_parser.cup Outdated
RESULT = new DropPartitionClause(ifExists, partitionName, isTempPartition);
RESULT = new DropPartitionClause(ifExists, partitionName, isTempPartition, !isTempPartition);
:}
| KW_DROPP opt_tmp:isTempPartition KW_PARTITION opt_if_exists:ifExists ident:partitionName

@chaoylichaoyliJul 7, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DROPP is a not a good syntax. It's better to use

DROP TABLE $table_name FORCE

@caiconghuicaiconghuiJul 27, 2020

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chaoyli I has fixed it before. old commit version?

Comment threadfe/src/main/java/org/apache/doris/catalog/Catalog.java
@chaoyli
chaoyli merged commit 150f8e0 into apache:masterJul 28, 2020
This was referenced Jul 29, 2020
Closed
Merged
morningman pushed a commit that referenced this pull request Jul 30, 2020
@caiconghui
[Bug] Fix fe meta version problem, make drop meta check code easy to …
237271c
…read and add doc content for drop meta check (#4205)
This PR is mainly do three things:
1. Fix fe meta version bug introduced by #4029 , when fix conflict with #4086 2. Make drop check code easy to read
3. Add doc content for drop meta check
@caiconghui
caiconghui deleted the drop_check branch July 30, 2020 02:20
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chaoylichaoylichaoyli approved these changes
@morningmanmorningmanAwaiting requested review from morningman

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@caiconghui@morningman@chaoyli