Skip to content

HBASE-24740 Enable journal logging for HBase snapshot operation - #2104

Merged
bharathv merged 2 commits into
apache:masterfrom
sguggilam:master
Jul 21, 2020
Merged

HBASE-24740 Enable journal logging for HBase snapshot operation#2104
bharathv merged 2 commits into
apache:masterfrom
sguggilam:master

Conversation

@sguggilam

Copy link
Copy Markdown
Contributor

No description provided.

@Apache-HBase

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec7m 7sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall4m 7smaster passed
+1 💚checkstyle1m 12smaster passed
+1 💚spotbugs2m 8smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 44sthe patch passed
-0 ⚠️checkstyle1m 10shbase-server: The patch generated 1 new + 22 unchanged - 3 fixed = 23 total (was 25)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 17sPatch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚spotbugs2m 16sthe patch passed
_ Other Tests _
+1 💚asflicense0m 13sThe patch does not generate ASF License warnings.
41m 53s
SubsystemReport/Notes
DockerClient=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#2104
Optional Testsdupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
unameLinux 1045a40fa5e9 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / a857c6d
checkstylehttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count84 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@sguggilam

Copy link
Copy Markdown
ContributorAuthor

@bharathv Please review and merge

@bharathvbharathv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits, but lgtm, can commit once fixed. @sguggilam Can you please confirm that you did a quick sanity check on the journal logging? (given there are no tests).

false);
} else {
LOG.debug("Convert to Single Snapshot Manifest");
LOG.debug("Convert to Single Snapshot Manifest for " + this.desc.getName());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: parameterized logging

if (master.getTableStateManager().isTableState(snapshotTable,
TableState.State.ENABLED)) {
LOG.debug("Table enabled, starting distributed snapshot.");
LOG.debug("Table enabled, starting distributed snapshots for {}",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This still requires the isDebugEnabled() guard because ClientSnapshotDescriptionUtils.toString(..) is not a trivial call.

else if (master.getTableStateManager().isTableState(snapshotTable,
TableState.State.DISABLED)) {
LOG.debug("Table is disabled, running snapshot entirely on master.");
LOG.debug("Table is disabled, running snapshot entirely on master for {}",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above.

@sguggilam

Copy link
Copy Markdown
ContributorAuthor

Yes @bharathv , tested out the journal log entry and below is the snippet

Table snapshot journal :
Running FLUSH table snapshot emp1 C_M_SNAPSHOT_TABLE on table emp at 1595282584365
Consolidate snapshot: emp1 at 1595282584593
Loading Region manifests for emp1 at 1595282584593
Writing data manifest for emp1 at 1595282584603
Verifying snapshot: emp1 at 1595282584616

@bharathvbharathv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, thanks. Will merge it once the precommit is green.

@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 29sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 44smaster passed
+1 💚compile0m 55smaster passed
+1 💚shadedjars5m 35sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 40smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 29sthe patch passed
+1 💚compile0m 55sthe patch passed
+1 💚javac0m 55sthe patch passed
+1 💚shadedjars5m 40spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 35sthe patch passed
_ Other Tests _
-1 ❌unit147m 4shbase-server in the patch failed.
171m 3s
SubsystemReport/Notes
DockerClient=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#2104
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 40458fdb34fc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / a857c6d
Default Java1.8.0_232
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/testReport/
Max. process+thread count4124 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@bharathv
bharathv merged commit 430602a into apache:masterJul 21, 2020
bharathv pushed a commit that referenced this pull request Jul 21, 2020
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 430602a)
bharathv pushed a commit that referenced this pull request Jul 21, 2020
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 430602a)
bharathv pushed a commit that referenced this pull request Jul 21, 2020
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 430602a)
@Apache-HBase

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 55sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall5m 1smaster passed
+1 💚compile1m 14smaster passed
+1 💚shadedjars6m 33sbranch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 45shbase-server in master failed.
_ Patch Compile Tests _
+1 💚mvninstall4m 55sthe patch passed
+1 💚compile1m 21sthe patch passed
+1 💚javac1m 21sthe patch passed
+1 💚shadedjars7m 9spatch has no errors when building our shaded downstream artifacts.
-0 ⚠️javadoc0m 51shbase-server in the patch failed.
_ Other Tests _
-1 ❌unit199m 28shbase-server in the patch failed.
231m 5s
SubsystemReport/Notes
DockerClient=19.03.12 Server=19.03.12 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#2104
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 08a73f07b4ec 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / a857c6d
Default Java2020-01-14
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadochttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
unithttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/testReport/
Max. process+thread count3719 (vs. ulimit of 12500)
modulesC: hbase-server U: hbase-server
Console outputhttps://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-2104/1/console
versionsgit=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered byApache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
…he#2104)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 430602a)
wchevreuil pushed a commit to wchevreuil/hbase that referenced this pull request May 24, 2021
…he#2104)
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
(cherry picked from commit 430602a)
(cherry picked from commit 4e5ec22)
Change-Id: Ibe837047e9f93c608ec13972a9f6b1ac3622c92f
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@sguggilam@Apache-HBase@bharathv