Skip to content

PHOENIX-6197 AggregateIT and StoreNullsIT hangs - #933

Closed
richardantal wants to merge 3 commits into
apache:4.xfrom
richardantal:PHOENIX-6197
Closed

PHOENIX-6197 AggregateIT and StoreNullsIT hangs#933
richardantal wants to merge 3 commits into
apache:4.xfrom
richardantal:PHOENIX-6197

Conversation

@richardantal

Copy link
Copy Markdown
Contributor

No description provided.


// In this class we depend on the major compaction to be done instantly
// so wo are overwriting the PHOENIX_MAX_LOOKBACK_AGE_CONF_KEY to 0
@BeforeClass

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect the change in TestUtil to fix this case as well.
Why do we need to override PHOENIX_MAX_LOOKBACK_AGE_CONF_KEY here ?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My change in TestUtil solves the hanging tests, because there we waited until a put and delete is deleted, which never happened because they were just done before calling the major compaction with a PHOENIX_MAX_LOOKBACK_AGE of an hour.
In StoreNullsIT in testDeletes, we upsert a line into a table, delete it run the major compaction.
Then open a historical connection after the upsert but before the delete, previously when PHOENIX_MAX_LOOKBACK_AGE_CONF_KEY was set to 0 we could not read back the data, but if it is set to an hour, the major compaction will not delete that. So the test would fail.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the comment here is not accurate, thats right.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec6m 14sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ 4.x Compile Tests _
+1 💚mvninstall11m 44s4.x passed
+1 💚compile0m 59s4.x passed
+1 💚checkstyle0m 30s4.x passed
+1 💚javadoc0m 46s4.x passed
+0 🆗spotbugs3m 8sphoenix-core in 4.x has 956 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall6m 6sthe patch passed
+1 💚compile1m 0sthe patch passed
+1 💚javac1m 0sthe patch passed
+1 💚checkstyle0m 31sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 46sthe patch passed
+1 💚spotbugs3m 20sthe patch passed
_ Other Tests _
-1 ❌unit174m 49sphoenix-core in the patch failed.
+1 💚asflicense0m 31sThe patch does not generate ASF License warnings.
212m 53s
ReasonTests
Failed junit testsphoenix.end2end.TenantSpecificViewIndexSaltedIT
phoenix.end2end.index.ImmutableIndexExtendedIT
phoenix.end2end.StoreNullsIT
phoenix.end2end.index.GlobalImmutableTxIndexIT
phoenix.end2end.join.HashJoinPersistentCacheIT
phoenix.end2end.IndexRepairRegionScannerIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-933/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#933
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 0bc745df6e7a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / 2ff920a
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-933/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-933/1/testReport/
Max. process+thread count5930 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-933/1/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec1m 52sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
+1 💚test4tests0m 0sThe patch appears to include 1 new or modified test files.
_ 4.x Compile Tests _
+1 💚mvninstall12m 35s4.x passed
+1 💚compile1m 6s4.x passed
+1 💚checkstyle0m 33s4.x passed
+1 💚javadoc0m 55s4.x passed
+0 🆗spotbugs3m 57sphoenix-core in 4.x has 954 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall7m 33sthe patch passed
+1 💚compile1m 13sthe patch passed
+1 💚javac1m 13sthe patch passed
+1 💚checkstyle0m 36sthe patch passed
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 58sthe patch passed
+1 💚spotbugs4m 8sthe patch passed
_ Other Tests _
-1 ❌unit192m 3sphoenix-core in the patch failed.
+1 💚asflicense0m 30sThe patch does not generate ASF License warnings.
230m 36s
ReasonTests
Failed junit testsphoenix.end2end.UngroupedIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-933/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#933
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux 6cd439b8389a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / afda3c1
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-933/2/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-933/2/testReport/
Max. process+thread count5582 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-933/2/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@gjacoby126gjacoby126 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thanks for tracking this down.


Put put = new Put(markerRowKey);
put.addColumn(QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES, QueryConstants.EMPTY_COLUMN_VALUE_BYTES, QueryConstants.EMPTY_COLUMN_VALUE_BYTES);
long timestamp = 0;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 0L

@stoty

Copy link
Copy Markdown
Contributor

Addressed Geoffrey's NIT, and committed to fix the build ASAP.

@stotystoty closed this Oct 23, 2020
@stoty

Copy link
Copy Markdown
Contributor

StoreNullsIT fails when running as part of the full IT suite.
Probably needs to be switched to @NeedsOwnCluster

@stotystoty reopened this Oct 23, 2020
@stoty

stoty commented Oct 25, 2020

Copy link
Copy Markdown
Contributor

Opened a PR for the addendum at #939 so that the precommit tests get executed on it.

@gjacoby126

Copy link
Copy Markdown
Contributor

JIRA is resolved, closing PR

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@richardantal@stoty@gjacoby126