Skip to content

HBASE-27543 May be miss data when use mob - #4935

Open
WenFeiYi wants to merge 1 commit into
apache:masterfrom
WenFeiYi:master
Open

HBASE-27543 May be miss data when use mob#4935
WenFeiYi wants to merge 1 commit into
apache:masterfrom
WenFeiYi:master

Conversation

@WenFeiYi

@WenFeiYiWenFeiYi commented Dec 21, 2022

Copy link
Copy Markdown
Contributor

No description provided.

@Apache-HBase

Apache-HBase commented Dec 21, 2022

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 8sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall2m 51smaster passed
+1 💚compile2m 25smaster passed
+1 💚checkstyle0m 38smaster passed
+1 💚spotless0m 44sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 30smaster passed
_ Patch Compile Tests _
+1 💚mvninstall2m 29sthe patch passed
+1 💚compile2m 27sthe patch passed
+1 💚javac2m 27sthe patch passed
+1 💚checkstyle0m 35sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck8m 52sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚spotless0m 41spatch has no errors when running spotless:check.
+1 💚spotbugs1m 34sthe patch passed
_ Other Tests _
+1 💚asflicense0m 11sThe patch does not generate ASF License warnings.
31m 48s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4935
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux 34afa6986f09 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 222ec68
Default JavaEclipse Adoptium-11.0.17+8
Max. process+thread count79 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Dec 21, 2022

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 59sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 40smaster passed
+1 💚compile1m 4smaster passed
+1 💚shadedjars5m 30sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 34smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 8sthe patch passed
+1 💚compile0m 49sthe patch passed
+1 💚javac0m 49sthe patch passed
+1 💚shadedjars5m 32spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 35sthe patch passed
_ Other Tests _
-1 ❌unit222m 50shbase-server in the patch failed.
250m 8s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4935
Optional Testsjavac javadoc unit shadedjars compile
unameLinux dfa8ab5ace91 5.4.0-1085-aws #92~18.04.1-Ubuntu SMP Wed Aug 31 17:21:08 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 222ec68
Default JavaTemurin-1.8.0_352-b08
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/testReport/
Max. process+thread count2433 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

apurtell
apurtell approved these changes Jan 17, 2023
View reviewed changes

@apurtellapurtell left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change seems fine.
Is it possible to add a unit test?

Apache9
Apache9 requested changes Jan 18, 2023
View reviewed changes
Comment threadhbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
fd.latestPutTs =
long latestPutTs =
tmp == null ? HConstants.LATEST_TIMESTAMP : TimeRangeTracker.parseFrom(tmp).getMax();
fd.latestPutTs = Math.max(fd.latestPutTs, latestPutTs);

@Apache9Apache9Jan 18, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check the code, we do not assign any value to fd.latestPutTs, so what is the actual problem here? The latestPutTs info for bulk loaded files can not negative and cause problem?

@Apache-HBase

Apache-HBase commented Jan 30, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 57sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall5m 25smaster passed
+1 💚compile2m 51smaster passed
+1 💚checkstyle0m 38smaster passed
+1 💚spotless0m 48sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 46smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 44sthe patch passed
+1 💚compile2m 54sthe patch passed
+1 💚javac2m 54sthe patch passed
-0 ⚠️checkstyle0m 37shbase-server: The patch generated 1 new + 7 unchanged - 1 fixed = 8 total (was 8)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck19m 20sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚spotless0m 52spatch has no errors when running spotless:check.
+1 💚spotbugs2m 3sthe patch passed
_ Other Tests _
+1 💚asflicense0m 15sThe patch does not generate ASF License warnings.
53m 9s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4935
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux fc05fb47f83c 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 382681e
Default JavaEclipse Adoptium-11.0.17+8
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count82 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jan 30, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 54sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall3m 39smaster passed
+1 💚compile0m 48smaster passed
+1 💚shadedjars4m 26sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 27smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 17sthe patch passed
+1 💚compile0m 48sthe patch passed
+1 💚javac0m 48sthe patch passed
+1 💚shadedjars4m 27spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26sthe patch passed
_ Other Tests _
+1 💚unit211m 21shbase-server in the patch passed.
234m 52s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4935
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 153f71c1335e 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 382681e
Default JavaEclipse Adoptium-11.0.17+8
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/testReport/
Max. process+thread count2488 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jan 30, 2023

Copy link
Copy Markdown

🎊 +1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 1sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 38smaster passed
+1 💚compile0m 49smaster passed
+1 💚shadedjars5m 20sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 33smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 12sthe patch passed
+1 💚compile0m 55sthe patch passed
+1 💚javac0m 55sthe patch passed
+1 💚shadedjars5m 25spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 28sthe patch passed
_ Other Tests _
+1 💚unit207m 54shbase-server in the patch passed.
235m 7s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4935
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a1c829675bbe 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 382681e
Default JavaTemurin-1.8.0_352-b08
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/testReport/
Max. process+thread count2641 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jan 30, 2023

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec0m 53sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 7smaster passed
+1 💚compile1m 0smaster passed
+1 💚shadedjars4m 52sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 27smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 52sthe patch passed
+1 💚compile0m 54sthe patch passed
+1 💚javac0m 54sthe patch passed
+1 💚shadedjars5m 8spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 30sthe patch passed
_ Other Tests _
-1 ❌unit14m 16shbase-server in the patch failed.
37m 47s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR#4935
Optional Testsjavac javadoc unit shadedjars compile
unameLinux a30499c26dff 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 382681e
Default JavaEclipse Adoptium-11.0.17+8
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/testReport/
Max. process+thread count590 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jan 30, 2023

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 7sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
_ master Compile Tests _
+1 💚mvninstall3m 50smaster passed
+1 💚compile2m 27smaster passed
+1 💚checkstyle0m 34smaster passed
+1 💚spotless0m 41sbranch has no errors when running spotless:check.
+1 💚spotbugs1m 23smaster passed
_ Patch Compile Tests _
+1 💚mvninstall3m 12sthe patch passed
+1 💚compile2m 24sthe patch passed
+1 💚javac2m 24sthe patch passed
-0 ⚠️checkstyle0m 32shbase-server: The patch generated 1 new + 7 unchanged - 1 fixed = 8 total (was 8)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚hadoopcheck12m 52sPatch does not cause any errors with Hadoop 3.2.4 3.3.4.
-1 ❌spotless0m 34spatch has 63 errors when running spotless:check, run spotless:apply to fix.
+1 💚spotbugs1m 29sthe patch passed
_ Other Tests _
+1 💚asflicense0m 9sThe patch does not generate ASF License warnings.
38m 57s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#4935
Optional Testsdupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
unameLinux 29109ac13629 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 382681e
Default JavaEclipse Adoptium-11.0.17+8
checkstylehttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
spotlesshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count81 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/console
versionsgit=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase

Apache-HBase commented Jan 30, 2023

Copy link
Copy Markdown

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 36sDocker mode activated.
-0 ⚠️yetus0m 3sUnprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚mvninstall4m 39smaster passed
+1 💚compile0m 52smaster passed
+1 💚shadedjars5m 4sbranch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 26smaster passed
_ Patch Compile Tests _
+1 💚mvninstall4m 0sthe patch passed
+1 💚compile0m 52sthe patch passed
+1 💚javac0m 52sthe patch passed
+1 💚shadedjars5m 30spatch has no errors when building our shaded downstream artifacts.
+1 💚javadoc0m 25sthe patch passed
_ Other Tests _
-1 ❌unit13m 55shbase-server in the patch failed.
39m 20s
SubsystemReport/Notes
DockerClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR#4935
Optional Testsjavac javadoc unit shadedjars compile
unameLinux 616343ce17ae 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev-support/hbase-personality.sh
git revisionmaster / 382681e
Default JavaTemurin-1.8.0_352-b08
unithttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Resultshttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/testReport/
Max. process+thread count935 (vs. ulimit of 30000)
modulesC: hbase-server U: hbase-server
Console outputhttps://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/console
versionsgit=2.34.1 maven=3.8.6
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@apurtell
apurtell self-requested a review March 28, 2023 04:56
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Apache9Apache9Apache9 requested changes
@apurtellapurtellAwaiting requested review from apurtell

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@WenFeiYi@Apache-HBase@apurtell@Apache9