[SPARK-7288] Suppress compiler warnings due to use of sun.misc.Unsafe; add facade in front of Unsafe; remove use of Unsafe.setMemory

[JoshRosen]

This patch suppresses compiler warnings due to our use of sun.misc.Unsafe (introduced in #5725). These warnings can only be suppressed via the -XDignore.symbol.file javac flag; the @SuppressWarnings annotation won't work for these.

In order to restrict uses of this compiler flag to the unsafe module, I placed a facade in front of Unsafe so that other modules won't call it directly. This facade also will also help us to avoid accidental usage of deprecated Unsafe methods or methods that aren't supported in Java 6.

I also removed an unnecessary use of Unsafe.setMemory, which isn't present in certain versions of Java 6, and excluded the new unsafe module from Javadoc.

[JoshRosen]

Citation for the compiler flags:

• https://stackoverflow.com/questions/4065401/using-internal-sun-classes-with-javac
• https://stackoverflow.com/questions/12745088/how-to-suppress-the-proprietary-api-warning-for-sun-reflect-reflection

[rxin]

Maybe you've already figured it out, but is it possible to suppress warning only for the unsafe module?

[tgravescs]

trying it out

[JoshRosen]

Won't we get warnings in other modules that call PlatformDependent.UNSAFE.* methods, like UnsafeRow in SQL?

[JoshRosen]

I think that we'll be able to restrict this compiler flag to the unsafe module if we place a facade in front of sun.misc.Unsafe so that other modules don't directly call those methods. This should be pretty easy, so I'll take a shot at it now.

[tgravescs]

it built fine for me with jdk6 and this patch.

[JoshRosen]

Also checking to see whether I need to change unidoc compiler options...

[JoshRosen]

Alright, pushed a new commit that puts a facade in front of Unsafe and reduced the scope of the compiler flag to only affect the unsafe module. I'm not super-proficient with Maven, so it would be good for someone to double-check that the way I'm merging in the additional compiler argument configs is correct.

[SparkQA]

Test build #31442 has finished for PR 5814 at commit 50230c0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.
• This patch does not change any dependencies.

[JoshRosen]

When I ran my one of my aggregation hash map benchmarks with -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining enabled it looks like the static methods in the UNSAFE facade are being inlined. I grepped the output for "org.apache.spark.unsafe.PlatformDependent$UNSAFE::" and it looks like all call sites are being inlined. @rxin, is this a methodologically sound way to check that the facade didn't add any overhead?

[rxin]

LGTM.

[SparkQA]

Test build #31447 has finished for PR 5814 at commit 9e8c483.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • public static final class UNSAFE
• This patch does not change any dependencies.