Uh oh!
There was an error while loading. Please reload this page.
RATIS-1882. Support read-after-write consistency - #913
Merged
Conversation
szetszwo
approved these changes
Sep 1, 2023
Contributor
There was a problem hiding this comment.
+1 the change looks good. Just a very minor comment inlined.
Also, spent some time to add another unit test; see
https://issues.apache.org/jira/secure/attachment/13062665/TestReadAfterWrite.patch
Could you include it?
Comment on lines
+62
to
+65
| final CompletableFuture<Long> future = ref != null? ref.get(): null; | ||
| if (future != null) { | ||
| return future; | ||
| } |
Contributor
There was a problem hiding this comment.
Just found that these few lines can be simplified as below.
if (ref != null) {
returnref.get();
}szetszwo
reviewed
Sep 1, 2023
| @Override | ||
| public CompletableFuture<RaftClientReply> sendReadAfterWrite(Message message) { | ||
| return send(RaftClientRequest.readAfterWriteConsistentRequestType(), message, null); |
Contributor
There was a problem hiding this comment.
@SzyWilliam , Should sendReadAfterWrite use UnorderedAsync.send(..)? Indeed, all the linearizable reads should use UnorderedAsync.
MemberAuthor
There was a problem hiding this comment.
That's true! Thanks for pointing out, I'll change all the linearizable reads to use UnorderedAsync
SzyWilliam
commented
Sep 4, 2023
MemberAuthor
@szetszwo Thanks very much for the patch! It's good to know |
szetszwo
reviewed
Sep 4, 2023
szetszwo
left a comment
Contributor
There was a problem hiding this comment.
Let's use OrderedAsync due to the test failure. Could you also include the following?
diff --gita/ratis-common/src/main/java/org/apache/ratis/protocol/RaftClientRequest.javab/ratis-common/src/main/java/org/apache/ratis/protocol/RaftClientRequest.javaindex4b0c5166..220694ce100644
--- a/ratis-common/src/main/java/org/apache/ratis/protocol/RaftClientRequest.java
+++ b/ratis-common/src/main/java/org/apache/ratis/protocol/RaftClientRequest.java@@ -227,7 +227,10@@publicclassRaftClientRequestextendsRaftClientMessage {
caseMESSAGESTREAM:
returntoString(getMessageStream());
caseREAD:
- return"RO";
+ finalReadRequestTypeProtoread = getRead();
+ returnread.getReadAfterWriteConsistent()? "RaW"
+ : read.getPreferNonLinearizable()? "RO(pNL)"
+ : "RO";
caseSTALEREAD:
return"StaleRead(" + getStaleRead().getMinIndex() + ")";
caseWATCH:
diff --gita/ratis-examples/src/test/java/org/apache/ratis/server/impl/TestReadAfterWrite.javab/ratis-examples/src/test/java/org/apache/ratis/server/impl/TestReadAfterWrite.javaindexf8f2405c..f515628c100644
--- a/ratis-examples/src/test/java/org/apache/ratis/server/impl/TestReadAfterWrite.java
+++ b/ratis-examples/src/test/java/org/apache/ratis/server/impl/TestReadAfterWrite.java@@ -128,8 +128,10@@publicclassTestReadAfterWriteLOG.info("readAfterWrite.get");
try {
// readAfterWrite should time out
- readAfterWrite.get(100, TimeUnit.MILLISECONDS);
- Assert.fail();
+ finalRaftClientReplyreply = readAfterWrite.get(100, TimeUnit.MILLISECONDS);
+ finalDoubleValueresult = (DoubleValue) Expression.Utils.bytes2Expression(
+ reply.getMessage().getContent().toByteArray(), 0);
+ Assert.fail("result=" + result + ", reply=" + reply);
} catch (TimeoutExceptione) {
LOG.info("Good", e);
}| @Override | ||
| public CompletableFuture<RaftClientReply> sendReadAfterWrite(Message message) { | ||
| return send(RaftClientRequest.readAfterWriteConsistentRequestType(), message, null); | ||
| return UnorderedAsync.send(RaftClientRequest.readAfterWriteConsistentRequestType(), message, null, client); |
Contributor
There was a problem hiding this comment.
Sorry, it actually not working since UnorderedAsync use a different channel. It can arrive the server earlier than the previous write request and get a small readIndex. TestReadAfterWrite will fail.
- Using unordered async
2023-09-04 11:16:59,888 DEBUG server.RaftServer$Division (LeaderStateImpl.java:getReadIndex(1099)) - readIndex=1, readAfterWriteConsistentIndex=1
- Using ordered async
2023-09-04 11:11:36,917 DEBUG server.RaftServer$Division (LeaderStateImpl.java:getReadIndex(1099)) - readIndex=3, readAfterWriteConsistentIndex=3
MemberAuthor
There was a problem hiding this comment.
You are right. Just investigated on OrderedAsync and UnorderedAsync and now I know why we should choose OrderedAsync. Thanks very much for your time spent and the detailed explanation!
Brokenice0415 pushed a commit
to Brokenice0415/ratis
that referenced
this pull request
Sep 6, 2023
tisonkun
commented
Sep 10, 2023
Member
Cool! I'm checking this out. |
tisonkun
commented
Sep 10, 2023
Member
May I ask what is the target release version of this feature? |
szetszwo
commented
Sep 12, 2023
Contributor
We should include this to both 3.0.0 and the next branch 2 releases. |
symious pushed a commit
to symious/ratis
that referenced
this pull request
Mar 20, 2024
RexXiong pushed a commit
to apache/celeborn
that referenced
this pull request
May 30, 2024
### What changes were proposed in this pull request? Bump Ratis version from 2.5.1 to 3.0.1. Address incompatible changes: - RATIS-589. Eliminate buffer copying in SegmentedRaftLogOutputStream.(apache/ratis#964) - RATIS-1677. Do not auto format RaftStorage in RECOVER.(apache/ratis#718) - RATIS-1710. Refactor metrics api and implementation to separated modules. (apache/ratis#749) ### Why are the changes needed? Bump Ratis version from 2.5.1 to 3.0.1. Ratis has released v3.0.0, v3.0.1, which release note refers to [3.0.0](https://ratis.apache.org/post/3.0.0.html), [3.0.1](https://ratis.apache.org/post/3.0.1.html). The 3.0.x version include new features like pluggable metrics and lease read, etc, some improvements and bugfixes including: - 3.0.0: Change list of ratis 3.0.0 In total, there are roughly 100 commits diffing from 2.5.1 including: - Incompatible Changes - RaftStorage Auto-Format - RATIS-1677. Do not auto format RaftStorage in RECOVER. (apache/ratis#718) - RATIS-1694. Fix the compatibility issue of RATIS-1677. (apache/ratis#731) - RATIS-1871. Auto format RaftStorage when there is only one directory configured. (apache/ratis#903) - Pluggable Ratis-Metrics (RATIS-1688) - RATIS-1689. Remove the use of the thirdparty Gauge. (apache/ratis#728) - RATIS-1692. Remove the use of the thirdparty Counter. (apache/ratis#732) - RATIS-1693. Remove the use of the thirdparty Timer. (apache/ratis#734) - RATIS-1703. Move MetricsReporting and JvmMetrics to impl. (apache/ratis#741) - RATIS-1704. Fix SuppressWarnings(“VisibilityModifier”) in RatisMetrics. (apache/ratis#742) - RATIS-1710. Refactor metrics api and implementation to separated modules. (apache/ratis#749) - RATIS-1712. Add a dropwizard 3 implementation of ratis-metrics-api. (apache/ratis#751) - RATIS-1391. Update library dropwizard.metrics version to 4.x (apache/ratis#632) - RATIS-1601. Use the shaded dropwizard metrics and remove the dependency (apache/ratis#671) - Streaming Protocol Change - RATIS-1569. Move the asyncRpcApi.sendForward(..) call to the client side. (apache/ratis#635) - New Features - Leader Lease (RATIS-1864) - RATIS-1865. Add leader lease bound ratio configuration (apache/ratis#897) - RATIS-1866. Maintain leader lease after AppendEntries (apache/ratis#898) - RATIS-1894. Implement ReadOnly based on leader lease (apache/ratis#925) - RATIS-1882. Support read-after-write consistency (apache/ratis#913) - StateMachine API - RATIS-1874. Add notifyLeaderReady function in IStateMachine (apache/ratis#906) - RATIS-1897. Make TransactionContext available in DataApi.write(..). (apache/ratis#930) - New Configuration Properties - RATIS-1862. Add the parameter whether to take Snapshot when stopping to adapt to different services (apache/ratis#896) - RATIS-1930. Add a conf for enable/disable majority-add. (apache/ratis#961) - RATIS-1918. Introduces parameters that separately control the shutdown of RaftServerProxy by JVMPauseMonitor. (apache/ratis#950) - RATIS-1636. Support re-config ratis properties (apache/ratis#800) - RATIS-1860. Add ratis-shell cmd to generate a new raft-meta.conf. (apache/ratis#901) - Improvements & Bug Fixes - Netty - RATIS-1898. Netty should use EpollEventLoopGroup by default (apache/ratis#931) - RATIS-1899. Use EpollEventLoopGroup for Netty Proxies (apache/ratis#932) - RATIS-1921. Shared worker group in WorkerGroupGetter should be closed. (apache/ratis#955) - RATIS-1923. Netty: atomic operations require side-effect-free functions. (apache/ratis#956) - RaftServer - RATIS-1924. Increase the default of raft.server.log.segment.size.max. (apache/ratis#957) - RATIS-1892. Unify the lifetime of the RaftServerProxy thread pool (apache/ratis#923) - RATIS-1889. NoSuchMethodError: RaftServerMetricsImpl.addNumPendingRequestsGauge apache/ratis#922 (apache/ratis#922) - RATIS-761. Handle writeStateMachineData failure in leader. (apache/ratis#927) - RATIS-1902. The snapshot index is set incorrectly in InstallSnapshotReplyProto. (apache/ratis#933) - RATIS-1912. Fix infinity election when perform membership change. (apache/ratis#954) - RATIS-1858. Follower keeps logging first election timeout. (apache/ratis#894) - 3.0.1:This is a bugfix release. See the [changes between 3.0.0 and 3.0.1](apache/ratis@ratis-3.0.0...ratis-3.0.1) releases. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Cluster manual test. Closes#2480 from SteNicholas/CELEBORN-1400. Authored-by: SteNicholas <programgeek@163.com> Signed-off-by: Shuang <lvshuang.xjs@alibaba-inc.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Support read-after-write consistency.
Thanks @szetszwo very much for the patch. Only made minor changes and added unit tests to the patch.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/RATIS-1882
How was this patch tested?
New unit tests.