Just me or Gradle tests fail for `iceberg-hive` when env variable `HIVE_HOME` is present?

[fbocse]

$ hive --version
...
Hive 3.1.1
Git git://daijymacpro-2.local/Users/daijy/commit/hive -r f4e0529634b6231a0072295da48af466cf2f10b7
Compiled by daijy on Tue Oct 23 17:19:24 PDT 2018
From source with checksum 6deca5a8401bbb6c6b49898be6fcb80e

$ ls -las $HIVE_HOME
/usr/local/hive/current -> apache-hive-3.1.1-bin

$ which hive
/usr/local/hive/current/bin/hive

So when running ./gradlew test it starts yielding errors

> Task :iceberg-hive:test

com.netflix.iceberg.hive.HiveTablesTest > testFailure FAILED
 javax.jdo.JDOFatalInternalException at HiveTablesTest.java:43
 Caused by: java.lang.reflect.InvocationTargetException at HiveTablesTest.java:43
 Caused by: org.datanucleus.exceptions.NucleusException at HiveTablesTest.java:43
 Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException at HiveTablesTest.java:43

com.netflix.iceberg.hive.HiveTablesTest > testExistingTableUpdate FAILED
 javax.jdo.JDOFatalInternalException at HiveTablesTest.java:43
 Caused by: java.lang.reflect.InvocationTargetException at HiveTablesTest.java:43
 Caused by: org.datanucleus.exceptions.NucleusException at HiveTablesTest.java:43
 Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException at HiveTablesTest.java:43

com.netflix.iceberg.hive.HiveTablesTest > testConcurrentFastAppends FAILED
 javax.jdo.JDOFatalInternalException at HiveTablesTest.java:43
 Caused by: java.lang.reflect.InvocationTargetException at HiveTablesTest.java:43
 Caused by: org.datanucleus.exceptions.NucleusException at HiveTablesTest.java:43
 Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException at HiveTablesTest.java:43

com.netflix.iceberg.hive.HiveTablesTest > testCreate FAILED
 javax.jdo.JDOFatalInternalException at HiveTablesTest.java:43
 Caused by: java.lang.reflect.InvocationTargetException at HiveTablesTest.java:43
 Caused by: org.datanucleus.exceptions.NucleusException at HiveTablesTest.java:43
 Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException at HiveTablesTest.java:43

4 tests completed, 4 failed

FAILURE: Build failed with an exception.

Workaround:
Just set HIVE_HOME env var to empty string (fix in shell or just override it in code maybe setup method ?)

[kbendick]

@fbocse I just attempted to reproduce this issue on my macbook with no success. I have never experienced this issue, but I've also never hived much outside of a real hadoop cluster.

TLDR: You might have already tried this, but have you tried ./gradlew clean?

I had hive 2.x installed locally, but HIVE_HOME was not set. I ran ./gradlew test and everything succeeded. I then brew upgrade hive to ensure I was on the same version as you and then set HIVE_HOME to attempt to replicate this.

> brew upgrade hive
.... get coffee
> hive --version

# Manually setting it as homebrew install didn't set it for me.
> HIVE_HOME=/usr/local/bin/hive
> echo $HIVE_HOME
/usr/local/bin/hive
> ls -las $HIVE_HOME
.... /usr/local/bin/hive -> ../Cellar/hive/3.1.1/bin/hive

# Verified we're on the same version
> hive --version
SLF4J: Typical classpath shenanigans.
....
Hive 3.1.1
Git git://daijymacpro-2.local/Users/daijy/commit/hive -r f4e0529634b6231a0072295da48af466cf2f10b7
Compiled by daijy on Tue Oct 23 17:19:24 PDT 2018
From source with checksum 6deca5a8401bbb6c6b49898be6fcb80e

# Check that all of my previous tests passed
> ./gradlew test
33 actionable tasks: 33 up-to-date

# Clean and retest everything, 
> echo $HIVE_HOME && ./gradlew clean test
/usr/local/bin/hive

# Some gradle output, which included 4/4 passing tests from :iceberg-hive
> Configure project : ...
... pet the dog ....
BUILD SUCCESSFUL in 3m 32s
33 actionable tasks: 33 executed

This might be due to differences in our environments, such as in how our hive installations were performed etc. But a good old ./gradlew clean might help out if you haven't tried it. I'm not super familiar with gradle myself (mostly I've used pants or bazel), so ignore this advice if it was painfully obvious already :)

[kbendick]

Is it possible that you've edited your hive-site.xml? I know that there are some datanucleus configurations in hive-site.xml.

For completeness sake, I've verified also that my local hive installation is working. I'm using a local mysql as my metastore backend (as configured in hadoop's hive-site.xml)

> hive -e "create table test_table(c1 string);"
....
OK
Time taken: 1.526 seconds

> hive -e "show tables;"
...
OK
test_table

[rdblue]

Thanks for trying to reproduce, @kbendick. I'm going to close this, but feel free to reopen if it turns out to be a problem.