HIVE-29709: Add health check to docker

[ayushtkn]

What changes were proposed in this pull request?

Add health check to Hive docker startup

Why are the changes needed?

Better user experiance

Does this PR introduce any user-facing change?

Yes, post hive cluster startup, you need not to wait randomly

How was this patch tested?

Manually

Before

[+] Building 0.0s (0/0)
[+] Running 7/7
✔ Network hive Created 0.0s
✔ Volume "hive_hive-db" Created 0.0s
✔ Volume "hive_warehouse" Created 0.0s
✔ Volume "hive_scratch" Created 0.0s
✔ Container postgres Started 1.4s
✔ Container metastore Started 1.1s
✔ Container hiveserver2 Started 1.1s
ayushsaxena@Q3NW54Y0C5 docker % docker exec -it hiveserver2 beeline -u 'jdbc:hive2://hiveserver2:10000/'
Connecting to jdbc:hive2://hiveserver2:10000/
26/07/06 14:22:15 [main]: WARN jdbc.HiveConnection: Failed to connect to hiveserver2:10000
Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status. Enable verbose error messages (--verbose=true) for more information.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://hiveserver2:10000/: java.net.ConnectException: Connection refused (state=08S01,code=0)
ayushsaxena@Q3NW54Y0C5 docker % docker exec -it hiveserver2 beeline -u 'jdbc:hive2://hiveserver2:10000/'
Connecting to jdbc:hive2://hiveserver2:10000/
26/07/06 14:22:20 [main]: WARN jdbc.HiveConnection: Failed to connect to hiveserver2:10000
Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status. Enable verbose error messages (--verbose=true) for more information.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://hiveserver2:10000/: java.net.ConnectException: Connection refused (state=08S01,code=0)
ayushsaxena@Q3NW54Y0C5 docker % docker exec -it hiveserver2 beeline -u 'jdbc:hive2://hiveserver2:10000/'
Connecting to jdbc:hive2://hiveserver2:10000/
26/07/06 14:22:25 [main]: WARN jdbc.HiveConnection: Failed to connect to hiveserver2:10000
Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status. Enable verbose error messages (--verbose=true) for more information.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://hiveserver2:10000/: java.net.ConnectException: Connection refused (state=08S01,code=0)
ayushsaxena@Q3NW54Y0C5 docker % docker exec -it hiveserver2 beeline -u 'jdbc:hive2://hiveserver2:10000/'
Connecting to jdbc:hive2://hiveserver2:10000/
26/07/06 14:22:37 [main]: WARN jdbc.HiveConnection: Failed to connect to hiveserver2:10000
Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status. Enable verbose error messages (--verbose=true) for more information.
Error: Could not open client transport with JDBC Uri: jdbc:hive2://hiveserver2:10000/: java.net.ConnectException: Connection refused (state=08S01,code=0)

After - Non-LLAP

[+] Building 0.0s (0/0)
[+] Running 7/7
✔ Network hive Created 0.0s
✔ Volume "hive_scratch" Created 0.0s
✔ Volume "hive_hive-db" Created 0.0s
✔ Volume "hive_warehouse" Created 0.0s
✔ Container postgres Healthy 17.2s
✔ Container metastore Healthy 17.2s
✔ Container hiveserver2 Healthy 27.2s
Hive cluster is ready.
ayushsaxena@Q3NW54Y0C5 docker % docker exec -it hiveserver2 beeline -u 'jdbc:hive2://hiveserver2:10000/'
Connecting to jdbc:hive2://hiveserver2:10000/
Connected to: Apache Hive (version 4.3.0-SNAPSHOT)
Driver: Hive JDBC (version 4.3.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 4.3.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://hiveserver2:10000/>

After - LLAP

+] Building 0.0s (0/0)
[+] Running 14/14
✔ Network hive Created 0.0s
✔ Volume "zookeeper_datalog" Created 0.0s
✔ Volume "zookeeper_logs" Created 0.0s
✔ Volume "hive_hive-db" Created 0.0s
✔ Volume "hive_warehouse" Created 0.0s
✔ Volume "hive_scratch" Created 0.0s
✔ Volume "zookeeper_data" Created 0.0s
✔ Container zookeeper Healthy 17.5s
✔ Container postgres Healthy 17.5s
✔ Container hive-llapdaemon-2 Healthy 17.4s
✔ Container tezam Healthy 17.4s
✔ Container hive-llapdaemon-1 Healthy 17.4s
✔ Container metastore Healthy 17.4s
✔ Container hiveserver2 Healthy 27.4s
Hive cluster is ready.
ayushsaxena@Q3NW54Y0C5 docker % docker exec -it hiveserver2 beeline -u 'jdbc:hive2://hiveserver2:10000/'
Connecting to jdbc:hive2://hiveserver2:10000/
Connected to: Apache Hive (version 4.3.0-SNAPSHOT)
Driver: Hive JDBC (version 4.3.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 4.3.0-SNAPSHOT by Apache Hive
0: jdbc:hive2://hiveserver2:10000/>

[abstractdog]

very nice improvement @ayushtkn , 2 comments only

[abstractdog]

for clarity's sake, zookeeper is supposed to return "imok", however this happens after start:

docker compose exec zookeeper sh -c 'printf ruok | nc -w 2 127.0.0.1 2181'
...
ruok is not executed because it is not in the whitelist.

maybe

environment:
ZOO_4LW_COMMANDS_WHITELIST: ruok

I think if the zookeeper is unable to start, then a connection refused is already enough to ensure the unhealthy state, however, it would be good to somehow have "imok" in the response check

[ayushtkn]

done

[abstractdog]

is there a chance to have healthcheck for tezams?
tez.am.rpc.port could be set to a static value, and multiple tez AM's won't collide given they are on different hosts like llap daemons

https://github.com/apache/tez/blob/4bccf3a2339c913ade0bc5447bdfc505c5536128/tez-dag/src/main/java/org/apache/tez/dag/api/client/DAGClientServer.java#L69

[ayushtkn]

have added tez am port check, hardcoded one

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[ayushtkn]

Tested after the changes

[abstractdog]

works perfectly, thanks @ayushtkn, LGTM