[SPARK-25535][core] Work around bad error handling in commons-crypto.

[vanzin]

The commons-crypto library does some questionable error handling internally,
which can lead to JVM crashes if some call into native code fails and cleans
up state it should not.

While the library is not fixed, this change adds some workarounds in Spark code
so that when an error is detected in the commons-crypto side, Spark avoids
calling into the library further.

Tested with existing and added unit tests.

[SparkQA]

Test build #96637 has finished for PR 22557 at commit fe3f06b.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• trait BaseErrorHandler extends Closeable
• class ErrorHandlingReadableChannel(

[SparkQA]

Test build #96639 has finished for PR 22557 at commit 4606ed9.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[vanzin]

@squito

[squito]

unrelated, but shouldn't EncryptedMessage.transferTo() not keep looping if target.write() doesn't accept all the data?

[squito]

since you're throwing a mock IOException above, this would be a little more clear if here you checked the message too

assert(intercept[IOException] {
errorHandler.read(out)
}.getMessage() === ("Cipher stream is closed."))

(you can also use assertThrows if you're not looking at the exception at all, though doesn't really matter)

[squito]

its a little confusing that there are two closeables. I'd add a comment here that wrapped is what the cipherStream is already wrapping, and we want to make sure it gets closed if the cipherStream has an internal error.

(when I see a variable named wrapped, I assumed it was what this was wrapping, not what the cipherStream was wrapping)

[squito]

any particular reason to set these to null, rather than having an isValid flag in here? you'd get an NPE if you ever tried to use the TranportCipher after this -- you are protecting against that in the wrapping code, but seems you could do it here.

[vanzin]

I can add a check that the ciphers are not null, but no need to have a separate flag. These need to be set to null so that close knows not to try to mess with them.

[vanzin]

Actually the check is already there I'll just add a message to it.

[SparkQA]

Test build #96714 has finished for PR 22557 at commit e6e5382.

• This patch fails Java style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[vanzin]

retest this please

[SparkQA]

Test build #96720 has finished for PR 22557 at commit e6e5382.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[vanzin]

unrelated, but shouldn't EncryptedMessage.transferTo() not keep looping if target.write() doesn't accept all the data?

Yes I noticed that too, but separate change. Hopefully I can fix other stuff in commons-crypto that would allow more of that code to be cleaned up...

[squito]

lgtm

will leave for a day before mergning

[vanzin]

Ping

[squito]

merged to master

[dongjoon-hyun]

Hi, All.
Can we have this in branch-2.4 too since Spark 2.4.x is the last version of 2.x lines which we need to support for a long time?
cc @dbtsai

[dbtsai]

+1

[vanzin]

Sure.

[vanzin]

#24476

[dongjoon-hyun]

Thank you so much, @vanzin and @dbtsai !