Uh oh!
There was an error while loading. Please reload this page.
HBASE-25549 Provide a switch that allows avoiding reopening all regions when modifying a table to prevent RIT storms. - #2924
Merged
Conversation
Apache-HBase
commented
Feb 4, 2021
💔 -1 overall
This message was automatically generated. |
GeorryHuangforce-pushed
the
lazy_alter
branch
2 times, most recently
from
February 4, 2021 11:06
0f54edb to
14ca0e9CompareApache-HBase
commented
Feb 4, 2021
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 4, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 4, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 4, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 4, 2021
🎊 +1 overall
This message was automatically generated. |
busbey
commented
Feb 4, 2021
Contributor
Why a new command rather than an optional lazy flag for the existing command? |
Apache-HBase
commented
Feb 4, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Feb 4, 2021
🎊 +1 overall
This message was automatically generated. |
GeorryHuang
commented
Feb 5, 2021
ContributorAuthor
@busbey Actually, I followed up the way of command 'alter_async' do, which added a so-called lazy flag to the existing function 'admin.alter()' of Ruby. The command 'alter_async' also show up in Ruby as a "new command". I guess 'alter_async' followed some guidelines? |
bharathv
commented
Feb 10, 2021
Contributor
@GeorryHuang I added a comment in the jira. IMO its good to roll "async" behavior into the actual alter command implementation. The original commit that added it is a decade old so I'm guessing very few people around here have any context of it. |
GeorryHuangforce-pushed
the
lazy_alter
branch
4 times, most recently
from
April 1, 2021 09:08
0928fa8 to
b59b96cCompareApache-HBase
commented
Apr 1, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 1, 2021
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 1, 2021
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 1, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 1, 2021
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 1, 2021
💔 -1 overall
This message was automatically generated. |
GeorryHuangforce-pushed
the
lazy_alter
branch
2 times, most recently
from
April 1, 2021 23:54
4716b7f to
e27220aCompareApache-HBase
commented
Apr 2, 2021
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Apr 2, 2021
💔 -1 overall
This message was automatically generated. |
gourabtaparia
approved these changes
Oct 16, 2023
gourabtaparia
left a comment
There was a problem hiding this comment.
LGTM.
I am relatively new to contributing to HBase code, so will wait for other members also to have a look.
virajjasani
commented
Oct 18, 2023
Contributor
Sorry for the delay and thanks for working on this @GeorryHuang, let me come back to the review soon as i can. @bbeaudreault if the changes look good to you in the meantime, please feel free to take it forward :) |
| hbase> alter 't1', CONFIGURATION => {'hbase.hregion.scan.loadColumnFamiliesOnDemand' => 'true'} | ||
| hbase> alter 't1', {NAME => 'f2', CONFIGURATION => {'hbase.hstore.blockingStoreFiles' => '10'}} | ||
| You can also set configuration setting with REOPEN_REGIONS=>'false' to avoid regions RIT, which |
Contributor
There was a problem hiding this comment.
Can we also explain the type of changes that don't allow reopen=false, such as adding/removing CFs or changing table name?
ContributorAuthor
There was a problem hiding this comment.
Sure, I've made the changes
Comment on lines
+124
to
+125
| 0 != this.unmodifiedTableDescriptor.getTableName() | ||
| .compareTo(this.modifiedTableDescriptor.getTableName()) |
| this(env, newTableDescriptor, latch, oldTableDescriptor, shouldCheckDescriptor, true); | ||
| } | ||
| public ModifyTableProcedure(final MasterProcedureEnv env, |
Contributor
There was a problem hiding this comment.
since this class is IA.Private, it'd be preferable to not add a new constructor overload unless absolutely necessary. Are there few enough callers that you could simply update one of the existing constructors?
ContributorAuthor
There was a problem hiding this comment.
Removed the new constructor I added before and altered the existing constructor instead.
Uh oh!
There was an error while loading. Please reload this page.
Apache-HBase
commented
Nov 1, 2023
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 1, 2023
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 1, 2023
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 1, 2023
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 1, 2023
💔 -1 overall
This message was automatically generated. |
| private long modifyTable(final TableName tableName, | ||
| final TableDescriptorGetter newDescriptorGetter, final long nonceGroup, final long nonce, | ||
| final boolean shouldCheckDescriptor) throws IOException { | ||
| return modifyTable(tableName, newDescriptorGetter, nonceGroup, nonce, shouldCheckDescriptor, |
Contributor
There was a problem hiding this comment.
I was looking into why we need this overload. I see this is called from addColumn/deleteColumn/modifyColumn. Does it make sense to add reopenRegion support to modifyColumn at least?
ContributorAuthor
There was a problem hiding this comment.
Now we have two methods:
modifyTable(..[five parameters]..) modifyTable(..[five parameters].., boolean reopenRegions)
The methods 'addColumn/deleteColumn/modifyColumn' that you mentioned will call method modifyTable(..[five parameters]..) directly as before. They are not aware of the 'reopenRegions' parameter.
To make the code structure cleaner, I only changed one line in method modifyTable(..[five parameters]..), which then turns into modifyTable(..[five parameters].., boolean reopenRegions). So now, you can see that modifyTable(..[five parameters]..) calls modifyTable(..[five parameters].., boolean reopenRegions) with reopenRegions=true.
bbeaudreault
approved these changes
Nov 1, 2023
bbeaudreault
left a comment
Contributor
There was a problem hiding this comment.
LGTM. I do wonder whether we should add modifyColumn support here, but not a blocker.
bbeaudreault
approved these changes
Nov 1, 2023
bbeaudreault
left a comment
Contributor
There was a problem hiding this comment.
LGTM. I do wonder whether we should add modifyColumn support here, but not a blocker.
Apache-HBase
commented
Nov 1, 2023
💔 -1 overall
This message was automatically generated. |
Apache9
commented
Nov 3, 2023
Contributor
We are good to go? |
virajjasani
approved these changes
Nov 6, 2023
| } | ||
| } | ||
| private boolean isTablePropertyModified(TableDescriptor oldDescriptor, |
Contributor
There was a problem hiding this comment.
nit: javadoc would be great
GeorryHuang
commented
Nov 8, 2023
ContributorAuthor
Yes, I am going to merge it into the master branch. |
Apache-HBase
commented
Nov 8, 2023
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 8, 2023
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 8, 2023
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 8, 2023
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 8, 2023
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Nov 8, 2023
💔 -1 overall
This message was automatically generated. |
bbeaudreault pushed a commit
that referenced
this pull request
Dec 1, 2023
…ns when modifying a table to prevent RIT storms. (#2924) Co-authored-by: Huangzhuoyue <huangzhuoyue@apache.org> Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org> Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org> Signed-off-by: Bharath Vissapragada <bharathv@apache.org> Signed-off-by: Esteban Gutierrez <esteban@apache.org> Reviewed-by: Sean Busbey <busbey@apache.org> Reviewed-by: Gourab Taparia
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Provide an optional lazy_mode for the alter command to modify the TableDescriptor without the region entering the RIT. The modification will take effect when the region is reopened.