Uh oh!
There was an error while loading. Please reload this page.
HBASE-28654 Support only remove expired files in the compact process - #6008
Open
wangxin97-02 wants to merge 1 commit into
Open
HBASE-28654 Support only remove expired files in the compact process#6008wangxin97-02 wants to merge 1 commit into
wangxin97-02 wants to merge 1 commit into
Conversation
Apache-HBase
commented
Jun 20, 2024
💔 -1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 20, 2024
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 20, 2024
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Jun 20, 2024
💔 -1 overall
This message was automatically generated. |
Apache9
commented
Aug 13, 2024
Contributor
I prefer we change the config name to something like 'compaction.skip-merging-compact', to indicate that this config is used to skip the real compaction operation. As even if we set this config to true, there are still other options which could make us not remove expired files, such as 'hbase.store.delete.expired.storefile'. We should carefully document these behaviors. |
Apache9
commented
Aug 13, 2024
Contributor
You can also send a discuss email to dev list, to see if there are better names for this config, since I'm not a native English speaker... |
wangxin97-02force-pushed
the
HBASE-28654
branch
from
September 4, 2024 11:58
cd969e0 to
a9f756bComparewangxin97-02
commented
Sep 4, 2024
ContributorAuthor
OK. I have changed and add logs for the config 'hbase.store.delete.expired.storefile' is false. |
Apache-HBase
commented
Sep 4, 2024
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 4, 2024
🎊 +1 overall
This message was automatically generated. |
| * true. If flag is false then do normal compactions. | ||
| * @return true if table compaction only remove expired file, skipping merge files. | ||
| */ | ||
| boolean isSkipMergingCompaction(); |
Contributor
There was a problem hiding this comment.
shouldSkipMergingCompaction make more sense to me over isSkipMergingCompaction
| if (this.region.getTableDescriptor().isSkipMergingCompaction()) { | ||
| if (!conf.getBoolean("hbase.store.delete.expired.storefile", true)) { | ||
| LOG.info("Because the conf of hbase.store.delete.expired.storefile is false, so the table " |
Contributor
There was a problem hiding this comment.
Can we rephrase as "Since config 'hbase.store.delete.expired.storefile' is set to false, ignoring SKIP_MERGING_COMPACTION set at table level"
| private static final Bytes COMPACTION_ENABLED_KEY = new Bytes(Bytes.toBytes(COMPACTION_ENABLED)); | ||
| /** | ||
| * Used by HBase Shell interface to access this metadata attribute which denotes if the table |
Contributor
There was a problem hiding this comment.
Are we planning to expose this via shell as another commit?
ContributorAuthor
There was a problem hiding this comment.
Now, we can use shell to change the table description with SKIP_MERGING_COMPACTION.
alter 'tableName',CONFIGURATION => {'SKIP_MERGING_COMPACTION' => 'true'}
| removeUnneededFiles(); | ||
| if (this.region.getTableDescriptor().isSkipMergingCompaction()) { | ||
| if (!conf.getBoolean("hbase.store.delete.expired.storefile", true)) { |
Contributor
There was a problem hiding this comment.
Also could you refactor 'hbase.store.delete.expired.storefile' into a variable along with its default value? I see only one usage of this in the code base, apart from testcode.
ContributorAuthor
There was a problem hiding this comment.
Also could you refactor 'hbase.store.delete.expired.storefile' into a variable along with its default value? I see only one usage of this in the code base, apart from testcode.
I have changed based on your suggestion. Please review again when you have time.
wangxin97-02force-pushed
the
HBASE-28654
branch
from
September 5, 2024 03:29
a9f756b to
80c7b26Compare
wangxin97-02force-pushed
the
HBASE-28654
branch
from
September 5, 2024 03:34
80c7b26 to
4665a54CompareApache-HBase
commented
Sep 5, 2024
🎊 +1 overall
This message was automatically generated. |
Apache-HBase
commented
Sep 5, 2024
💔 -1 overall
This message was automatically generated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/HBASE-28654