Skip to content

PHOENIX-6160 Simplifying concurrent mutation handling for global Indexes - #897

Closed
kadirozde wants to merge 2 commits into
apache:4.xfrom
kadirozde:6160
Closed

PHOENIX-6160 Simplifying concurrent mutation handling for global Indexes#897
kadirozde wants to merge 2 commits into
apache:4.xfrom
kadirozde:6160

Conversation

@kadirozde

Copy link
Copy Markdown
Contributor

No description provided.

@gjacoby126gjacoby126 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Algorithm makes sense, it's a nice simplification, and just had a few nits on the code. My bigger concern is the potential increase in tail latency on writes to hot rows in write latency-sensitive applications. Would be good to get some perf numbers.

public Put getNextDataRowState(ImmutableBytesPtr rowKeyPtr) {
Pair<Put, Put> rowState = dataRowStates.get(rowKeyPtr);
if (rowState != null) {
return dataRowStates.get(rowKeyPtr).getSecond();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rowState.getSecond(). No need to pull out of the hashmap a second time.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was the intention and missed it. I will make the change

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Algorithm makes sense, it's a nice simplification, and just had a few nits on the code. My bigger concern is the potential increase in tail latency on writes to hot rows in write latency-sensitive applications. Would be good to get some perf numbers.

Good suggestion. I will do some perf runs and update the Jira.

@gjacoby126, I have updated the design doc with the performance testing results ( https://docs.google.com/document/d/12H_MwsPtyM0ORiBHclBpBLZWtm4zpY_cc5y_pwtgMUk/edit#heading=h.yt8378ps0k6e)

private String dataTableName;

private static final int DEFAULT_ROWLOCK_WAIT_DURATION = 30000;
private static final int DEFAULT_CONCURRENT_MUTATION_WAIT_DURATION_IN_MS = 1000;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1s seems long

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will change it to 100ms and let me know if you have a different suggestion

// coprocessor calls. TODO: remove after HBASE-18127 when available

private static class BatchMutateContext {
private BatchMutatePhase currentPhase = BatchMutatePhase.PRE;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This variable gets accessed from multiple threads -- should it be atomic? Since only one thread will write to it at once, it would currently only occasionally prevent having to go through the wait loop more than times than necessary. Might also prevent correctness issues in the future if these assumptions change.

@kadirozdekadirozdeSep 29, 2020

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question! The concurrent batch of mutations is a set such that every pair of batches in this set has at least one common row. Since a BatchMutateContext object of a batch is modified only after the row locks for all the rows that are mutated by this batch are acquired, there can be only one thread can acquire the locks for its batch and safely access all the batch contexts in the set of concurrent batches. I will add this to doc and comment it on the code. Please note that the row state is also accessed. So, atomic variables are not necessary. Also, making currentPhase atomic and leaving the rest as it is will give a wrong impression.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks.


public void add() {
PendingRow(BatchMutateContext context) {
lastContext = context;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

slightly cleaner if we can initialize the count to 1 in the constructor itself.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do that

@kadirozde

Copy link
Copy Markdown
ContributorAuthor

Algorithm makes sense, it's a nice simplification, and just had a few nits on the code. My bigger concern is the potential increase in tail latency on writes to hot rows in write latency-sensitive applications. Would be good to get some perf numbers.

Good suggestion. I will do some perf runs and update the Jira.

@kadirozde

Copy link
Copy Markdown
ContributorAuthor

It seems I mistakenly closed this JIRA

@kadirozdekadirozde reopened this Sep 29, 2020
@stoty

Copy link
Copy Markdown
Contributor

💔 -1 overall

VoteSubsystemRuntimeComment
+0 🆗reexec6m 33sDocker mode activated.
_ Prechecks _
+1 💚dupname0m 0sNo case conflicting files found.
+1 💚hbaseanti0m 0sPatch does not have any anti-patterns.
+1 💚@author0m 0sThe patch does not contain any @author tags.
-1 ❌test4tests0m 0sThe patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ 4.x Compile Tests _
+1 💚mvninstall31m 39s4.x passed
+1 💚compile1m 0s4.x passed
+1 💚checkstyle0m 37s4.x passed
+1 💚javadoc0m 47s4.x passed
+0 🆗spotbugs3m 6sphoenix-core in 4.x has 957 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚mvninstall29m 24sthe patch passed
+1 💚compile0m 59sthe patch passed
+1 💚javac0m 59sthe patch passed
-1 ❌checkstyle0m 37sphoenix-core: The patch generated 64 new + 375 unchanged - 44 fixed = 439 total (was 419)
+1 💚whitespace0m 0sThe patch has no whitespace issues.
+1 💚javadoc0m 47sthe patch passed
+1 💚spotbugs3m 19sphoenix-core generated 0 new + 956 unchanged - 1 fixed = 956 total (was 957)
_ Other Tests _
-1 ❌unit174m 40sphoenix-core in the patch failed.
+1 💚asflicense0m 31sThe patch does not generate ASF License warnings.
256m 29s
ReasonTests
Failed junit testsphoenix.end2end.index.IndexMetadataIT
phoenix.end2end.IndexExtendedIT
phoenix.end2end.ConnectionUtilIT
phoenix.end2end.SkipScanQueryIT
phoenix.end2end.index.IndexCoprocIT
SubsystemReport/Notes
DockerClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-897/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR#897
Optional Testsdupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile
unameLinux fb953d384f5a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build toolmaven
Personalitydev/phoenix-personality.sh
git revision4.x / fc3d25c
Default JavaPrivate Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstylehttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-897/3/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
unithttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-897/3/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Resultshttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-897/3/testReport/
Max. process+thread count6238 (vs. ulimit of 30000)
modulesC: phoenix-core U: phoenix-core
Console outputhttps://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-897/3/console
versionsgit=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered byApache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@gjacoby126gjacoby126 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the perf test, glad to see the numbers look good. +1.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@kadirozde@stoty@tkhurana@gjacoby126