Skip to content

Always issue HELLO when the server should understand it, not just for RESP3 - #3175

Merged
mgravell merged 2 commits into
mainfrom
marc/always-hello
Aug 14, 2026
Merged

Always issue HELLO when the server should understand it, not just for RESP3#3175
mgravell merged 2 commits into
mainfrom
marc/always-hello

Conversation

@mgravell

Copy link
Copy Markdown
Collaborator

Fixes#2968 (the remaining concern), and fixes a v3 regression against twemproxy found on the way.

The #2968 problem

With INFO unavailable, discovery fell back to SET {guid} replica-read-only PX 1 NX to detect a read-only replica. That probe writes a random, unprefixable key, so it can never be allow-listed by an ACL key pattern - which is precisely the situation that made INFO unavailable in the first place (INFO and CONFIG are both @dangerous). The reporter was left with permanent ACL-log noise they had no way to silence.

HELLO's reply carries role, needs no key, and is in the @connection category. It also reports the version, mode (standalone/sentinel/cluster) and connection id. We were only issuing it when negotiating RESP3.

The change

  • ConfigurationOptions.TryHello(out int protocolVersion) alongside TryResp3(), plus RedisFeatures.Hello (6.0). HELLO is issued whenever it is available and the assumed server version is 6.0+. Two independent opt-outs: $hello= in the command map, and a sub-6.0 defaultVersion.
  • The key-based replica probe is skipped when HELLO is going to report the role (same handshake batch) or has already reported it (sticky per-endpoint flag, for later reconfigure sweeps).
  • Since ACLs are themselves 6.0+, the probe is now unreachable for anyone who can express key restrictions at all. Pre-6.0 servers keep the old fallback, where there are no key ACLs to upset.

The RESP2 HELLO is deliberately not the same message as the RESP3 one:

  • HELLO 3 stays first-in-pipeline and carries the credentials, as it must for RESP3 negotiation to work on a secured server.
  • HELLO 2 is a bareHELLO issued afterAUTH, on the interactive connection only. It negotiates nothing, and folding credentials in would change how credential failures surface - there is a pre-existing difference between AUTH failing on its own and AUTH failing inside HELLO, which is a separate bug and not one to inherit here.

Proxies (measured against live proxies, not assumed)

TargetVersionHELLO behaviour
twemproxy0.5.0 (newest release; 2021)parsed unsupported command 'hello', then closes the connection
envoyv1.31 (previous test pin)-unsupported command 'hello'
envoyv1.39 (current)HELLO 2proxied to an arbitrary backend node; HELLO 3 -> -NOPROTO

Consequences, both fixed here by excluding HELLO from CommandMap.Twemproxy and CommandMap.Envoyproxy:

  • v3 already broke twemproxy on default settings - a regression against v2. v2 assumed version 3.0, so TryResp3() was false and no HELLO was sent; v3 raised the assumed default to 6.0, and the twemproxy map did not exclude HELLO. Verified with a real twemproxy: SocketClosed (0-read), connection never usable.
  • Envoy 1.39's forwarded HELLO 2 reported version=8.9.241, mode=cluster, role=replica - i.e. some backend node, not the endpoint. mode/role from HELLO are therefore only applied to server types that support auto-configure, so proxies are excluded even if a hand-rolled command map re-enables the command.

Also bumps the test-topology envoy pin from v1.31 to v1.39 (8 minors stale, and the two behave differently here).

Tests

HelloHandshakeTests uses a recording in-process server (captures every command and its args) to pin exactly what the handshake issues:

  • HELLO at the right protocol level for both RESP2 and RESP3
  • no HELLO when disabled via the command map, or when defaultVersion is below 6.0
  • no replica probe when INFO/CONFIG are unavailable but HELLO is, with IsReplica coming from HELLO's role
  • the converse: HELLO unavailable too -> the probe is issued

Resp3HandshakeTests no longer skips the RESP2-client half of its matrix (128 -> 192 cases), so the "server does not understand HELLO" and "server insists on RESP2" spoofs now cover RESP2 clients too, and it asserts the negotiated protocol. Plus a CommandMapUnitTests case pinning the proxy exclusions.

Full suite green; EnvoyTests verified against a rebuilt v1.39 container.

Follow-ups (not in this PR)

… RESP3
Fixes#2968 (remaining concern).
When INFO is unavailable, discovery fell back to `SET {guid} replica-read-only PX 1 NX`
to detect a read-only replica. That probe writes a random, unprefixable key, so it can
never be allow-listed by an ACL key pattern - which is exactly the situation that makes
INFO unavailable in the first place (INFO and CONFIG are both @dangerous). HELLO's reply
carries "role", needs no key, and is in the @connection category.
So: issue HELLO whenever it is available and the assumed server version is 6.0+, and skip
the key-based probe when HELLO is going to tell us (or has already told us) the role. Two
independent opt-outs remain: `$hello=` in the command map, and a sub-6.0 `defaultVersion`.
As a bonus, RESP2 connections now learn the server version, mode and connection id from
the handshake as well.
The RESP2 HELLO is deliberately not the same message as the RESP3 one:
- `HELLO 3` stays first-in-pipeline and carries the credentials, as it must for RESP3
negotiation to happen at all on a secured server.
- `HELLO 2` is a bare HELLO issued *after* AUTH, on the interactive connection only. It
isn't negotiating anything, and folding credentials in would change how credential
failures surface (there is a pre-existing difference in behaviour between AUTH failing
on its own and AUTH failing inside HELLO; that is a separate bug, not one to inherit
here).
Also excludes HELLO from the twemproxy and envoyproxy command maps, and verifies against
both proxies locally:
- twemproxy 0.5.0 (the newest release) *closes the connection* on an unsupported command,
so HELLO is fatal. Since v3 raised the assumed default version to 6.0, RESP3 - and
therefore HELLO - became the default, and a default-configured twemproxy connection
never became usable at all. That is a v3 regression against v2, where the assumed
version of 3.0 meant no HELLO was ever sent.
- envoy up to ~1.31 answers "unsupported command"; 1.39 instead proxies HELLO to an
arbitrary backend node, so the version/role/mode in the reply describe some other
server. Taking that at face value would flip a proxy endpoint into cluster mode or mark
it as a replica, so HELLO's mode/role are now only applied to server types that support
auto-configure - proxies are excluded even if a hand-rolled command map re-enables the
command.
Bumps the test-topology envoy pin from v1.31 to v1.39 (8 minors stale, and the two
versions behave differently here).
Tests use a recording in-process server to pin exactly which commands the handshake
issues: HELLO at the right protocol level, absent when disabled either way, and the
replica probe present only when HELLO cannot tell us the role. The Resp3HandshakeTests
matrix no longer skips RESP2 clients (they issue HELLO now too), and asserts the
negotiated protocol.
EventId 110 is in use in another in-progress PR, and this doesn't warrant a
logger message of its own - the handshake HELLO is already visible in the
detail/parse logs.
@mgravell
mgravell merged commit 85cd5f6 into mainAug 14, 2026
6 checks passed
@mgravell
mgravell deleted the marc/always-hello branch August 14, 2026 12:32
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Uses "@dangerous" commands even when allowAdmin=false (default)

1 participant

@mgravell