Skip to content

HBASE-27659: Incremental backups should re-use splits from last full backup - #6370

Merged
ndimiduk merged 1 commit into
apache:masterfrom
hgromer:HBASE-27659
Jan 23, 2025
Merged

HBASE-27659: Incremental backups should re-use splits from last full backup#6370
ndimiduk merged 1 commit into
apache:masterfrom
hgromer:HBASE-27659

Conversation

@hgromer

Copy link
Copy Markdown
Contributor

No description provided.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@hgromer

Copy link
Copy Markdown
ContributorAuthor

The spotbugs issues seems unrelated

Code | Warning
-- | --
DMI | Random object created and used only once in org.apache.hadoop.hbase.util.LoadTestKVGenerator.getValueForRowColumn(int, byte[][])
| Bug type DMI_RANDOM_USED_ONLY_ONCE (click for details)In class org.apache.hadoop.hbase.util.LoadTestKVGeneratorIn method org.apache.hadoop.hbase.util.LoadTestKVGenerator.getValueForRowColumn(int, byte[][])Called method java.util.Random.nextBytes(byte[])At LoadTestKVGenerator.java:[line 111]
Bad practice Warnings
Code	Warning
DMI	Random object created and used only once in org.apache.hadoop.hbase.util.LoadTestKVGenerator.getValueForRowColumn(int, byte[][])
[Bug type DMI_RANDOM_USED_ONLY_ONCE (click for details)](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6370/6/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html#DMI_RANDOM_USED_ONLY_ONCE)
In class org.apache.hadoop.hbase.util.LoadTestKVGenerator
In method org.apache.hadoop.hbase.util.LoadTestKVGenerator.getValueForRowColumn(int, byte[][])
Called method java.util.Random.nextBytes(byte[])
At LoadTestKVGenerator.java:[line 111]

@Apache-HBase

This comment has been minimized.

@hgromer
hgromerforce-pushed the HBASE-27659 branch 2 times, most recently from e2c5386 to 69afa97CompareOctober 22, 2024 19:52
@hgromer

Copy link
Copy Markdown
ContributorAuthor

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@Apache-HBase

This comment has been minimized.

@ndimiduk

Copy link
Copy Markdown
Member
./hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestIncrementalBackup.java:105: try (Table ignored = TEST_UTIL.truncateTable(table1)) {:59: Must have at least one statement. [EmptyBlock]

well...

@ndimiduk

Copy link
Copy Markdown
Member

I guess checkstyle prefers

Tablet = null;
try {
t = TEST_UTIL.truncateTable(table1)
} finally {
if (t != null) {
t.close();
}
}

This is getting silly.

@ndimiduk

Copy link
Copy Markdown
Member

Let's go back to TEST_UTIL.truncateTable(table1).close() for these cases where an underlying exception will result in no closable making it up to the calling context.

@hgromer
hgromerforce-pushed the HBASE-27659 branch 2 times, most recently from dd8d6c6 to 43c21e4CompareJanuary 22, 2025 14:40
@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 27sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+0 🆗codespell0m 0scodespell was not available.
+0 🆗detsecrets0m 0sdetect-secrets was not available.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
_ master Compile Tests _
+0 🆗mvndep0m 18sMaven dependency ordering for branch
+1 💚mvninstall2m 49smaster passed
+1 💚compile3m 57smaster passed
+1 💚checkstyle0m 53smaster passed
+1 💚spotbugs2m 21smaster passed
+1 💚spotless0m 41sbranch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for patch
+1 💚mvninstall2m 48sthe patch passed
+1 💚compile3m 58sthe patch passed
-0 ⚠️javac0m 27s/results-compile-javac-hbase-backup.txthbase-backup generated 2 new + 102 unchanged - 0 fixed = 104 total (was 102)
+1 💚blanks0m 0sThe patch has no blanks issues.
+1 💚checkstyle0m 54sthe patch passed
+1 💚spotbugs2m 44sthe patch passed
+1 💚hadoopcheck10m 51sPatch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚spotless0m 41spatch has no errors when running spotless:check.
_ Other Tests _
+1 💚asflicense0m 24sThe patch does not generate ASF License warnings.
41m 3s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6370/29/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#6370
JIRA IssueHBASE-27659
Optional Testsdupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
unameLinux 860b4ad17fc9 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c8d87ac
Default JavaEclipse Adoptium-17.0.11+9
Max. process+thread count86 (vs. ulimit of 30000)
modulesC: hbase-server hbase-mapreduce hbase-backup U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6370/29/console
versionsgit=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeLogfileComment
+0 🆗reexec0m 28sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗mvndep0m 11sMaven dependency ordering for branch
+1 💚mvninstall2m 59smaster passed
+1 💚compile1m 31smaster passed
+1 💚javadoc0m 54smaster passed
+1 💚shadedjars5m 43sbranch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗mvndep0m 12sMaven dependency ordering for patch
+1 💚mvninstall2m 53sthe patch passed
+1 💚compile1m 31sthe patch passed
+1 💚javac1m 31sthe patch passed
+1 💚javadoc0m 53sthe patch passed
+1 💚shadedjars5m 38spatch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚unit243m 10shbase-server in the patch passed.
+1 💚unit25m 48shbase-mapreduce in the patch passed.
+1 💚unit18m 27shbase-backup in the patch passed.
316m 5s
SubsystemReport/Notes
DockerClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6370/29/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR#6370
JIRA IssueHBASE-27659
Optional Testsjavac javadoc unit compile shadedjars
unameLinux 0f3aff018244 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / c8d87ac
Default JavaEclipse Adoptium-17.0.11+9
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6370/29/testReport/
Max. process+thread count4694 (vs. ulimit of 30000)
modulesC: hbase-server hbase-mapreduce hbase-backup U: .
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6370/29/console
versionsgit=2.34.1 maven=3.9.8
Powered byApache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@ndimidukndimiduk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Thanks a lot for your patience, @hgromer .

@ndimiduk
ndimiduk merged commit f98621a into apache:masterJan 23, 2025
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Jan 23, 2025
…ackup (apache#6370)
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Jan 23, 2025
…ackup (apache#6370)
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org>
ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Jan 23, 2025
…ackup (apache#6370)
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org>
mnpoonia pushed a commit to mnpoonia/hbase that referenced this pull request Jan 24, 2025
…ackup (apache#6370)
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org>
mnpoonia pushed a commit to mnpoonia/hbase that referenced this pull request Jan 24, 2025
…ackup (apache#6370)
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org>
ndimiduk pushed a commit that referenced this pull request Jan 24, 2025
…ackup (#6370)
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org>
ndimiduk pushed a commit that referenced this pull request Jan 24, 2025
…ackup (#6370)
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org>
ndimiduk pushed a commit that referenced this pull request Jan 24, 2025
…ackup (#6370)
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org>
hgromer added a commit to HubSpot/hbase that referenced this pull request Feb 4, 2025
mokai87 pushed a commit to mokai87/hbase that referenced this pull request Aug 7, 2025
…ackup (apache#6370)
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
Signed-off-by: Ray Mattingly <rmattingly@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@hgromer@Apache-HBase@ndimiduk@rmdmattingly@DieterDP-ng