Uh oh!
There was an error while loading. Please reload this page.
HDDS-15768. Bracket IPv6 literals in Ratis peer addresses - #10686
Conversation
There was a problem hiding this comment.
Pull request overview
Fixes construction of Ratis peer target strings when the host is an IPv6 literal by ensuring the host:port form is unambiguous ([ipv6]:port). This aligns Ozone’s Ratis peer addressing with what gRPC/Netty target parsing expects and prevents failures caused by unbracketed IPv6 literals.
Changes:
- Added
HddsUtils.getHostPortString(host, port)to formathost:portwith IPv6 bracketing via GuavaHostAndPort. - Updated Ratis peer address generation in
RatisHelperand HA node peer address generation inNodeDetailsto use the new helper. - Added unit tests covering IPv6 bracketing behavior in both the utility and Ratis helper paths.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HddsUtils.java | Adds a shared helper to format host:port with correct IPv6 bracketing semantics. |
| hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/ratis/RatisHelper.java | Routes datanode Ratis peer address formatting through the new helper to support IPv6 literals. |
| hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/NodeDetails.java | Updates HA Ratis peer address formatting to use the new helper (avoids raw concatenation). |
| hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/TestHddsUtils.java | Adds focused unit coverage for getHostPortString across hostname/IPv4/IPv6 (bare and bracketed). |
| hadoop-hdds/common/src/test/java/org/apache/hadoop/hdds/ratis/TestRatisHelper.java | Adds a unit test validating IPv6 bracketing in RatisHelper.toRaftPeer(...) addresses. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
adoroszlai
left a comment
There was a problem hiding this comment.
Thanks @smengcl for the patch.
Uh oh!
There was an error while loading. Please reload this page.
adoroszlai
left a comment
There was a problem hiding this comment.
Thanks @smengcl for updating the patch.
(Please update your fork, either "Sync fork" via GitHub UI, or push master manually. This will trigger populate-cache, which should help with frequent rocksdb source downloaderrors. The tarball is normally stored in Maven cache.)
Uh oh!
There was an error while loading. Please reload this page.
smengcl
commented
Jul 10, 2026
Thanks @adoroszlai for reviewing and merging this!
Thanks for the tip! Just did. |
* master: (416 commits) HDDS-15808. Replace json-simple with Jackson in httpfsgateway (#10710) HDDS-15753. Rename TestClock to MockClock (#10712) HDDS-14759. Local Empty file is written and retained even when key write fails (#10707) HDDS-15647. Bump Node.js to 24.17.0 (#10590) HDDS-15783. Handle empty lists in ScmListCodec deserialization (#10690) HDDS-15768. Bracket IPv6 literals in Ratis peer addresses (#10686) HDDS-15679. [Recon] Fix DN dropdown mismatch with pending deletion API (#10615). HDDS-15692. Enable GitHub Copilot code review when a pull request is opened (#10705) HDDS-15739. Fix ListObjectsV2 continuation-token: wrong XML element name and empty-token handling (#10668) HDDS-15648. Improve TestSnapshotDiffManager#testThreadPoolIsFull drain scenario (#10632) HDDS-15803. Fix parameter number warning in OMKeyRequest.allocateBlock. (#10701) HDDS-15480. Potential for NPE / infinite loop in StreamBlockReader (#10431) HDDS-15523. Speed up TestOMDbCheckpointServletInodeBasedXfer (#10680) HDDS-15788. Fix SST filtering BOOTSTRAP_LOCK contention introduced by RocksDB 10 upgrade (#10692) HDDS-15752. Rename base tests and helpers for naming convention in OM (#10672) HDDS-15754. Rename TestableCluster to MockCluster (#10691) HDDS-15755. Bump develocity-maven-extension to 2.5.0 (#10674) HDDS-14956. Add S3 HAProxy to ozonesecure-ha/test-haproxy-s3g.sh (#10685) HDDS-15762. Remove mini-chaos-tests (#10682) HDDS-14577. Handle missing metadata dir when updating container state (#10565). ... Conflicts: hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneManagerVersion.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeStateMachine.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/states/endpoint/TestHeartbeatEndpointTask.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeManager.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeStateManager.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/safemode/SafeModeRuleFactory.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/upgrade/FinalizationStateManagerImpl.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestNodeStateManager.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestPendingContainerTracker.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/MockPipelineManager.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/TestSCMClientProtocolServer.java hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmKeyInfo.java hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/protocolPB/TestS3GrpcOmTransport.java hadoop-ozone/dist/src/main/compose/testlib.sh hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/docker-compose.yaml hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMRatisSnapshots.java hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/OMClientRequest.java hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerRequestHandler.java hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconStorageContainerManagerFacade.java
* HDDS-14496-zdu: (417 commits) HDDS-15808. Replace json-simple with Jackson in httpfsgateway (apache#10710) HDDS-15753. Rename TestClock to MockClock (apache#10712) HDDS-14759. Local Empty file is written and retained even when key write fails (apache#10707) HDDS-15647. Bump Node.js to 24.17.0 (apache#10590) HDDS-15783. Handle empty lists in ScmListCodec deserialization (apache#10690) HDDS-15768. Bracket IPv6 literals in Ratis peer addresses (apache#10686) HDDS-15679. [Recon] Fix DN dropdown mismatch with pending deletion API (apache#10615). HDDS-15692. Enable GitHub Copilot code review when a pull request is opened (apache#10705) HDDS-15739. Fix ListObjectsV2 continuation-token: wrong XML element name and empty-token handling (apache#10668) HDDS-15648. Improve TestSnapshotDiffManager#testThreadPoolIsFull drain scenario (apache#10632) HDDS-15803. Fix parameter number warning in OMKeyRequest.allocateBlock. (apache#10701) HDDS-15480. Potential for NPE / infinite loop in StreamBlockReader (apache#10431) HDDS-15523. Speed up TestOMDbCheckpointServletInodeBasedXfer (apache#10680) HDDS-15788. Fix SST filtering BOOTSTRAP_LOCK contention introduced by RocksDB 10 upgrade (apache#10692) HDDS-15752. Rename base tests and helpers for naming convention in OM (apache#10672) HDDS-15754. Rename TestableCluster to MockCluster (apache#10691) HDDS-15755. Bump develocity-maven-extension to 2.5.0 (apache#10674) HDDS-14956. Add S3 HAProxy to ozonesecure-ha/test-haproxy-s3g.sh (apache#10685) HDDS-15762. Remove mini-chaos-tests (apache#10682) HDDS-14577. Handle missing metadata dir when updating container state (apache#10565). ... Conflicts: hadoop-ozone/client/src/test/java/org/apache/hadoop/ozone/client/rpc/TestRpcClient.java
* HDDS-14496-zdu: (417 commits) HDDS-15808. Replace json-simple with Jackson in httpfsgateway (apache#10710) HDDS-15753. Rename TestClock to MockClock (apache#10712) HDDS-14759. Local Empty file is written and retained even when key write fails (apache#10707) HDDS-15647. Bump Node.js to 24.17.0 (apache#10590) HDDS-15783. Handle empty lists in ScmListCodec deserialization (apache#10690) HDDS-15768. Bracket IPv6 literals in Ratis peer addresses (apache#10686) HDDS-15679. [Recon] Fix DN dropdown mismatch with pending deletion API (apache#10615). HDDS-15692. Enable GitHub Copilot code review when a pull request is opened (apache#10705) HDDS-15739. Fix ListObjectsV2 continuation-token: wrong XML element name and empty-token handling (apache#10668) HDDS-15648. Improve TestSnapshotDiffManager#testThreadPoolIsFull drain scenario (apache#10632) HDDS-15803. Fix parameter number warning in OMKeyRequest.allocateBlock. (apache#10701) HDDS-15480. Potential for NPE / infinite loop in StreamBlockReader (apache#10431) HDDS-15523. Speed up TestOMDbCheckpointServletInodeBasedXfer (apache#10680) HDDS-15788. Fix SST filtering BOOTSTRAP_LOCK contention introduced by RocksDB 10 upgrade (apache#10692) HDDS-15752. Rename base tests and helpers for naming convention in OM (apache#10672) HDDS-15754. Rename TestableCluster to MockCluster (apache#10691) HDDS-15755. Bump develocity-maven-extension to 2.5.0 (apache#10674) HDDS-14956. Add S3 HAProxy to ozonesecure-ha/test-haproxy-s3g.sh (apache#10685) HDDS-15762. Remove mini-chaos-tests (apache#10682) HDDS-14577. Handle missing metadata dir when updating container state (apache#10565). ... Conflicts: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/metrics/SCMMetrics.java
* HDDS-14496-zdu: (416 commits) HDDS-15808. Replace json-simple with Jackson in httpfsgateway (apache#10710) HDDS-15753. Rename TestClock to MockClock (apache#10712) HDDS-14759. Local Empty file is written and retained even when key write fails (apache#10707) HDDS-15647. Bump Node.js to 24.17.0 (apache#10590) HDDS-15783. Handle empty lists in ScmListCodec deserialization (apache#10690) HDDS-15768. Bracket IPv6 literals in Ratis peer addresses (apache#10686) HDDS-15679. [Recon] Fix DN dropdown mismatch with pending deletion API (apache#10615). HDDS-15692. Enable GitHub Copilot code review when a pull request is opened (apache#10705) HDDS-15739. Fix ListObjectsV2 continuation-token: wrong XML element name and empty-token handling (apache#10668) HDDS-15648. Improve TestSnapshotDiffManager#testThreadPoolIsFull drain scenario (apache#10632) HDDS-15803. Fix parameter number warning in OMKeyRequest.allocateBlock. (apache#10701) HDDS-15480. Potential for NPE / infinite loop in StreamBlockReader (apache#10431) HDDS-15523. Speed up TestOMDbCheckpointServletInodeBasedXfer (apache#10680) HDDS-15788. Fix SST filtering BOOTSTRAP_LOCK contention introduced by RocksDB 10 upgrade (apache#10692) HDDS-15752. Rename base tests and helpers for naming convention in OM (apache#10672) HDDS-15754. Rename TestableCluster to MockCluster (apache#10691) HDDS-15755. Bump develocity-maven-extension to 2.5.0 (apache#10674) HDDS-14956. Add S3 HAProxy to ozonesecure-ha/test-haproxy-s3g.sh (apache#10685) HDDS-15762. Remove mini-chaos-tests (apache#10682) HDDS-14577. Handle missing metadata dir when updating container state (apache#10565). ... Conflicts: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerRequestHandler.java hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/upgrade/TestOMCancelPrepareRequest.java
Generated-by: Claude Code (Opus 4.8)
What changes were proposed in this pull request?
Ozone builds Ratis peer address strings by plainly concatenating a host and port (
host + ":" + port). When the host is an IPv6 literal (for example2001:db8::1), the result2001:db8::1:9858is ambiguous: it cannot be distinguished from a hostname, and neither Ratis nor gRPC can parse it. IPv6 literals must be wrapped in square brackets when combined with a port, e.g.[2001:db8::1]:9858.This affects every Ratis transport in Ozone:
RatisHelper.toRaftPeerAddress(used for theRATIS_SERVER,RATIS_ADMIN,RATIS, andRATIS_DATASTREAMpeer addresses).NodeDetails.getRatisHostPortStr(consumed byOzoneManagerRatisServer,SCMRatisServerImpl, andSCMHAManagerImplwhen callingRaftPeer.setAddress(...)).Ratis itself needs no change:
ratis-grpcpasses the peer address string verbatim toNettyChannelBuilder.forTarget(...), which accepts the bracketed[::1]:portform, and Ratis's ownNetUtils.createSocketAddr/address2Stringare already IPv6-aware. The defect is purely that Ozone hands Ratis unbracketed IPv6 strings.This PR adds a small bracket-aware formatter and routes the two Ratis peer-address producers through it:
HddsUtils.getHostPortString(String host, int port), which delegates to Guava'sHostAndPort.fromParts(host, port).toString(). It brackets bare and already-bracketed IPv6 literals, and leaves hostnames and IPv4 literals untouched.HddsUtilsalready depends on GuavaHostAndPort(used by the siblinggetHostPort), so no new dependency is introduced.RatisHelper.toRaftPeerAddressandNodeDetails.getRatisHostPortStrnow use the helper instead of raw string concatenation.Scope note: this PR fixes the Ratis peer-address path only. Other IPv6-unsafe address handling in Ozone (the
0.0.0.0bind defaults, the S3G/OFS URI and admin-CLI colon splitting, and otherhost + ":" + portsites such asGrpcOMFailoverProxyProvider) is tracked separately and is out of scope here.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15768
How was this patch tested?