HBASE-28767 Simplify backup bulk-loading code by DieterDP-ng · Pull Request #6134 · apache/hbase

DieterDP-ng · 2024-08-05T13:12:51Z

No functional changes.

Currently split into 2 commits as to help the reviewer with seeing what was changed. This can be squashed when merging.

ndimiduk

Much better than it was. thank you @DieterDP-ng

rmdmattingly · 2024-09-11T15:26:55Z

- }
-
 public static List<Delete> createDeleteForOrigBulkLoad(List<TableName> lst) {
+ // Unused???


should we act on this comment now?

Ill deal with it in HBASE-29003.

No functional changes.

No functional changes. Introduces the BulkLoad class as data container for the tracking of which files were bulk loaded (for backup purposes). This class could be a record class, but that would make backporting less straightforward.

DieterDP-ng · 2024-11-28T15:23:43Z

Apologies for letting this gather dust. Review comments have been addressed.

DieterDP-ng · 2024-12-02T15:50:02Z

💔 -1 overall
Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ docker 0m 4s Docker failed to build run-specific yetus/hbase:tp-18400}.
Subsystem Report/Notes
GITHUB PR #6134
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6134/2/console
versions git=2.17.1
Powered by Apache Yetus 0.15.0 https://yetus.apache.org
This message was automatically generated.

This looks a failure unrelated to the PR to me. Let me know if this is not the case.

ndimiduk

Thanks for addressing my nits. This is looking good by me. I've initiated another PR build since the last one had some infra-level failure.

Apache-HBase · 2024-12-13T13:08:27Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 28s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	3m 5s		master passed
+1 💚	compile	0m 29s		master passed
+1 💚	checkstyle	0m 10s		master passed
+1 💚	spotbugs	0m 32s		master passed
+1 💚	spotless	0m 44s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 56s		the patch passed
+1 💚	compile	0m 30s		the patch passed
+1 💚	javac	0m 30s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 7s	/buildtool-patch-checkstyle-hbase-backup.txt	The patch fails to run checkstyle in hbase-backup
+1 💚	spotbugs	0m 37s		the patch passed
+1 💚	hadoopcheck	10m 59s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚	spotless	0m 43s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 10s		The patch does not generate ASF License warnings.
		28m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6134/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6134
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 8c011f88d156 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `5932ae3`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6134/3/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-12-13T13:13:36Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	3m 13s		master passed
+1 💚	compile	0m 20s		master passed
+1 💚	javadoc	0m 15s		master passed
+1 💚	shadedjars	4m 54s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 39s		the patch passed
+1 💚	compile	0m 20s		the patch passed
+1 💚	javac	0m 20s		the patch passed
+1 💚	javadoc	0m 14s		the patch passed
+1 💚	shadedjars	4m 51s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	15m 35s		hbase-backup in the patch passed.
		33m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6134/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6134
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux f67eedb73dd9 5.4.0-200-generic #220-Ubuntu SMP Fri Sep 27 13:19:16 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `5932ae3`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6134/3/testReport/
Max. process+thread count	2972 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6134/3/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: DieterDP <90392398+DieterDP-ng@users.noreply.github.com>

Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

This comment has been minimized.
Sign in to view

DieterDP-ng commented Aug 6, 2024
View reviewed changes

Comment threadhbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupSystemTable.java Outdated

ndimiduk requested changes Sep 11, 2024
View reviewed changes

ndimiduk requested a review from rmdmattingly September 11, 2024 09:29

rmdmattingly reviewed Sep 11, 2024
View reviewed changes

DieterDP-ng added 2 commits November 28, 2024 16:11

HBASE-28767 Simplify backup bulk-loading code
343a11d
No functional changes.

HBASE-28767 Simplify backup bulk-loading code
5932ae3
No functional changes. Introduces the BulkLoad class as data container for the tracking of which files were bulk loaded (for backup purposes). This class could be a record class, but that would make backporting less straightforward.

DieterDP-ng force-pushed the HBASE-28767 branch from 3809779 to 5932ae3Compare November 28, 2024 15:22

DieterDP-ng requested review from ndimiduk and rmdmattingly November 28, 2024 17:37

This comment has been minimized.
Sign in to view

DieterDP-ng mentioned this pull request Dec 2, 2024
HBASE-29003 Proper bulk load tracking #6506
Merged

ndimiduk approved these changes Dec 13, 2024
View reviewed changes

ndimiduk merged commit 5f312dd into apache:masterDec 13, 2024

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Dec 13, 2024
HBASE-28767 Simplify backup bulk-loading code (apache#6134)
c1ed116
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Dec 13, 2024
HBASE-28767 Simplify backup bulk-loading code (apache#6134)
19643c3
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Dec 13, 2024
HBASE-28767 Simplify backup bulk-loading code (apache#6134)
9595552
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

ndimiduk pushed a commit that referenced this pull request Dec 13, 2024
HBASE-28767Simplify backup bulk-loading code (#6134)
430513a
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Dec 16, 2024
HBASE-28767 Simplify backup bulk-loading code (apache#6134)
706c989
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Dec 16, 2024
HBASE-28767 Simplify backup bulk-loading code (apache#6134)
4f26748
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

ndimiduk pushed a commit that referenced this pull request Dec 17, 2024
HBASE-28767Simplify backup bulk-loading code (#6134)
334accf
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

ndimiduk pushed a commit that referenced this pull request Dec 17, 2024
HBASE-28767Simplify backup bulk-loading code (#6134)
c3696a0
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

gvprathyusha6 pushed a commit to gvprathyusha6/hbase that referenced this pull request Dec 19, 2024
HBASE-28767 Simplify backup bulk-loading code (apache#6134)
886a104
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

hgromer pushed a commit to HubSpot/hbase that referenced this pull request Feb 3, 2025
HBASE-28767 Simplify backup bulk-loading code (apache#6134)
a57ba35
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

hgromer mentioned this pull request Feb 3, 2025
HubSpot Backport: HBASE-28767 Simplify backup bulk-loading code (#6134) HubSpot/hbase#143
Merged

hgromer added a commit to HubSpot/hbase that referenced this pull request Feb 3, 2025
HBASE-28767 Simplify backup bulk-loading code (apache#6134) (#143)
7341747
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: DieterDP <90392398+DieterDP-ng@users.noreply.github.com>

hgromer mentioned this pull request Feb 3, 2025
HubSpot Backport: HBASE-28767 Simplify backup bulk-loading code #6134 HubSpot/hbase#148
Merged

mokai87 pushed a commit to mokai87/hbase that referenced this pull request Aug 7, 2025
HBASE-28767 Simplify backup bulk-loading code (apache#6134)
74e8b90
Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>

Uh oh!

Conversation

DieterDP-ng commented Aug 5, 2024

Uh oh!

This comment has been minimized.

This comment has been minimized.

Uh oh!

ndimiduk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rmdmattinglySep 11, 2024

Choose a reason for hiding this comment

Uh oh!

DieterDP-ngNov 28, 2024

Choose a reason for hiding this comment

Uh oh!

DieterDP-ng commented Nov 28, 2024

Uh oh!

This comment has been minimized.

This comment has been minimized.

DieterDP-ng commented Dec 2, 2024

Uh oh!

ndimiduk left a comment

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Dec 13, 2024

Uh oh!

Apache-HBase commented Dec 13, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants