HIVE-29697: Add support for Persistable Sessions

[ayushtkn]

What changes were proposed in this pull request?

Add support for Persistable Sessions in Hive

Why are the changes needed?

In case of HS2 failures, the Session should be preserved for better HS2 HA

Does this PR introduce any user-facing change?

In case there is a HS2 crash, the other HS2 can reload the session, so session is preserved and the query retry doesn't fail with Invalid SessionHandle like

Error: Invalid SessionHandle: SessionHandle [5929b974-c356-4ebb-8ec4-e99071f6319d] (state=,code=0)

How was this patch tested?

UT, Manually for REDIS via K8s Operator
Killed the HS2 pod which had the session with

kubectl delete pod hive-hiveserver2-768f66c44d-tntnj --grace-period=0 --force

Checked running a query and if the temp table and configs set survived

[Copilot]

Pull request overview

Adds an initial implementation of “Persistable Sessions” for HiveServer2, allowing HS2 to recover session state (configs, jars, temp tables) from an external shared store (ZooKeeper/Redis) after failover.

Changes:

• Introduces a new service-session-store module with SessionStateStore plus ZooKeeper and Redis implementations and a JSON-serializable HiveSessionSnapshot.
• Integrates snapshot save/delete + session recovery logic into HS2 SessionManager, with a configurable fetch strategy.
• Adds utilities + hooks (PersistableSessionUtils, HiveSessionImpl notifications) and new unit/integration tests covering recovery and strategies.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
service/src/java/org/apache/hive/service/cli/session/SessionManager.java	Initializes the store, saves/deletes snapshots, and attempts session recovery/sync from the remote store.
service/src/java/org/apache/hive/service/cli/session/PersistableSessionUtils.java	Snapshot capture/hydration helpers and store save/delete helpers.
service/src/java/org/apache/hive/service/cli/session/HiveSessionProxy.java	Exposes the base session for unwrapping during snapshot capture.
service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java	Calls snapshot save notifications on detected “state-changing” commands and exposes captureSnapshot().
service/pom.xml	Adds dependency on the new session-store module.
service-session-store/src/main/java/org/apache/hive/service/cli/session/store/SessionStateStore.java	Defines the SPI for snapshot persistence.
service-session-store/src/main/java/org/apache/hive/service/cli/session/store/HiveSessionSnapshot.java	DTO for persisted session state (Jackson-serializable).
service-session-store/src/main/java/org/apache/hive/service/cli/session/store/ZooKeeperSessionStateStore.java	ZooKeeper-backed persistence implementation.
service-session-store/src/main/java/org/apache/hive/service/cli/session/store/RedisSessionStateStore.java	Redis-backed persistence implementation (TTL-based).
service-session-store/src/test/java/org/apache/hive/service/cli/session/store/TestSessionStateStoreBase.java	Base unit tests for store save/get/delete behavior.
service-session-store/src/test/java/org/apache/hive/service/cli/session/store/TestZooKeeperSessionStateStore.java	ZooKeeper store unit tests using Curator TestingServer.
service-session-store/src/test/java/org/apache/hive/service/cli/session/store/TestRedisSessionStateStore.java	Redis store unit tests using Testcontainers.
service-session-store/pom.xml	New module POM and dependencies (Jackson/Curator/Jedis).
pom.xml	Adds module + jedis.version property.
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestPersistableSessionBase.java	Integration tests that validate failover recovery + fetch strategies.
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestPersistableSessionWithZooKeeper.java	ZooKeeper-backed integration test wiring.
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestPersistableSessionWithRedis.java	Redis-backed integration test wiring.
itests/hive-unit/pom.xml	Adds test dependencies for the new module and Jedis.
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java	Adds new ConfVars for store class, strategy, and TTL.

Suppressed comments (1)

service/src/java/org/apache/hive/service/cli/session/PersistableSessionUtils.java:190

• When restoring temp tables during hydration, the warn log drops the exception stack trace (only logs e.getMessage()). That makes restore failures hard to debug, especially when a DDL fails due to missing jars/serde/permissions. Logging the exception object preserves useful context.

 for (Map.Entry<String, String> entry : snapshot.getTempTableDefinitions().entrySet()) {
try {
session.executeStatement(entry.getValue(), null);
} catch (Exception e) {
LOG.warn("Failed to restore temporary table {}: {}", entry.getKey(), e.getMessage());
}

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[tanishq-chugh]

Can we remove the error being logged here? On the console output, it initially appears as some error post query re-attempt.

[sonarqubecloud]

Quality Gate passed

Issues
48 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.3% Duplication on New Code

See analysis details on SonarQube Cloud

[tanishq-chugh]

Tried it out and works as expected 🚀

With ZK:

Initial Cluster state:

Created Temporary tables and changed session level configs:

HS2 to which session was connected crashes and new HS2 spins up in its place:

Same session accessible with temporary tables + session-configs:

With Redis:
New HS2 spins up after session-connected HS2 crashed:

Same session accessible with temporary tables + session-configs: