Fix native builds on GitHub Actions using Bazel cache

[karllessard]

This PR fixes most of the issues we are facing when building/releasing on GitHub Actions. Important updates are:

• Enable Bazel caching on a GCP bucket
• Better cleanup of Cuda resources

Bazel cache will reuse the same compiled binaries (at the file level) that were built in a previous workflow if the whole native tree (TF) is detected as being unchanged (it basically hashes the content of the files + its configuration). Multiple runs might be required after modifying the TF source tree since it can still take over 6 hours in some cases, but on the second run the build will restart from where it left.

After the cache is setup, a full build on Linux with GPU will take about 25 minutes instead of 6 hours. See this run

[Craigacp]

LGTM.

[karllessard]

Thanks @Craigacp , merging, let see how it goes with the deployment

[karllessard]

Bingo :) https://github.com/tensorflow/java/runs/2096897445?check_suite_focus=true

[rnett]

This broke local builds with build.sh: line 36: BUILD_EXTRA_FLAGS: unbound variable, BUILD_EXTRA_FLAGS doesn't seem to be sued by the CI anywhere though. Can we delete it?

[karllessard]

It's issued by the Maven plugin executing that script. Normally, you shouldn't run build.sh manually.

[rnett]

Yeah, I get that error when running maven install. Not sure why though, as that seems correct.

[karllessard]

Mmh, maybe the plugin don't expose the environment variable if it's empty? Possible, since it always has a value in the CI. Anyway I'm done building the artifacts of 0.3.0, which is not impacted by this other that users might have trouble if they want to rebuild 0.3.0 by their own... mmmh... ok I'll take a look this weekend, I might do a fix and rebuild everything again, I haven't published the artifacts so its fine, thanks