[SPARK-30642][ML][PYSPARK] LinearSVC blockify input vectors

[zhengruifeng]

What changes were proposed in this pull request?

1, stack input vectors to blocks (like ALS/MLP);
2, add new param blockSize;
3, add a new class InstanceBlock
4, standardize the input outside of optimization procedure;

Why are the changes needed?

1, reduce RAM to persist traing dataset; (save ~40% in test)
2, use Level-2 BLAS routines; (12% ~ 41% faster, without native BLAS)

Does this PR introduce any user-facing change?

a new param blockSize

How was this patch tested?

existing and updated testsuites

[zhengruifeng]

testCode:

importorg.apache.spark.ml.classification._importorg.apache.spark.storage.StorageLevelvardf= spark.read.format("libsvm").load("/data1/Datasets/a9a/a9a").withColumn("label", (col("label")+1)/2)
df.persist(StorageLevel.MEMORY_AND_DISK)
df.count
(0 until 8).foreach{ _ => df = df.union(df) }
df.count
newLinearSVC().setMaxIter(10).fit(df) // warm upvalsvc=newLinearSVC().setMaxIter(100)
valstart=System.currentTimeMillis; valmodel= svc.fit(df); valend=System.currentTimeMillis; end - start
valsvc=newLinearSVC().setMaxIter(100).setFitIntercept(false)
valstart=System.currentTimeMillis; valmodel= svc.fit(df); valend=System.currentTimeMillis; end - start

result:
this PR:
RAM:1418.9M
Duration: 396524(fitIntercept=true), 324944(fitIntercept=false)

MASTER:
RAM: 2.3G
Duration: 446354(fitIntercept=true), 441961(fitIntercept=false)

Native-BLAS is NOT used in above tests, maybe future performance gain can be obtained by setting appropriate Native BLAS.

[zhengruifeng]

Move the standardization outside of HingeAggregator, so that no longer need to standardize input in each iter.

[SparkQA]

Test build #117398 has finished for PR 27360 at commit 6dcb9eb.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• trait HasBlockSize extends Params
• class HasBlockSize(Params):

[SparkQA]

Test build #117400 has finished for PR 27360 at commit c51c37b.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

OK, I would generally support it if it's not a huge change, optimizes performance, and doesn't change behavior / correctness.

[zhengruifeng]

I added InstanceBlock as a common class, since I think it will also works in LoR, LiR, and other algs.
I add some testsuite to check correctness.
Thanks for reviewing! @srowen

[SparkQA]

Test build #117404 has finished for PR 27360 at commit ea83dc1.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #117407 has finished for PR 27360 at commit 7676b8e.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zhengruifeng]

retest this please

[SparkQA]

Test build #117410 has finished for PR 27360 at commit 7676b8e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #117416 has finished for PR 27360 at commit 487ee0f.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #117418 has finished for PR 27360 at commit 2f84a3d.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[zhengruifeng]

retest this please

[SparkQA]

Test build #117420 has finished for PR 27360 at commit 2f84a3d.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #117444 has finished for PR 27360 at commit cb91306.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #117474 has finished for PR 27360 at commit 819227e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zhengruifeng]

Merged to master.