ARROW-9156: [C++] Reducing the code size of the tensor module

[mrkn]

To reduce the size of libarrow.so, I'd like to reduce the size of tensor and sparse tensor codes.

TODO:

• Stop using template parameters in Tensor to SparseTensor converters
• Stop using template parameters in SparseTensor to Tensor converters

[github-actions]

https://issues.apache.org/jira/browse/ARROW-9156

[wesm]

This removes already more than 2MB of code from libarrow.so on Linux: great. I'll keep an eye on this

[mrkn]

@wesm Could you please review this?

[wesm]

Thank you for working on this, this is definitely very beneficial both for compilation time and code size. I left some minor comments about reducing logic duplication in a few places by factoring some things out into a helper function but otherwise this looks good, and the existing test suite gives good evidence that things are working as expected

I'm sensitive to the possible performance issues that this refactor may introduce (by changing inline functions to C function calls or functions with switch statements in them but per my comments within I think the best approach would be to identify the performance problems (if any), write benchmarks to illustrate them, and then pursue optimizations on a case by case basis.

[wesm]

You could also use an std::function for this, benchmarks would help show which is better. Either solution is going to be slower than the inlined version but if it is demonstrated to be a performance problem then we can revisit the template instantiation on a case by case basis to decide whether the tradeoff between code size and performance makes sense.

Additionally, in the future I think it could be beneficial to package the tensor support code in a libarrow_tensor in the future so that we could also "spin off" the tensor support into a separate pyarrow-tensor Python package so that users who do not need tensor support are not having to always carry around the compiled code that they may never use.

[wesm]

+1, modulo CI. Thank you for using the GetByteWidth helper function in so many places =)

[mrkn]

@wesm Is it better to work for benchmarking in other pull-request?

[wesm]

@mrkn it's up to you, it's fine with me if you work on performance tuning (or at least measurement) in another PR or this one

[mrkn]

@wesm OK. I continue to work for benchmarking in this pull-request. If I need more time to tune etc., I'll split the issue.

Awaiting benchmarking

[mrkn]

I wrote a benchmark code that measures the performance of conversion from Tensor to SparseTensor.
And I run this code with --repetitions=10 and got the following result.

• Converting to SparseCOOTensor is 1.4x-1.8x slower than the original.
• Converting to SparseCSRMatrix and SparseCSCMatrix is 1.0x-1.2x faster than the original.
• Converting to SparseCSFTensor is 1.0-1.3x slower than the original.

I don't think this result, especially SparseCOOTensor's case can be acceptable.
Now I'm trying to resolve this performance regression.

The full result is shown below:

Format	IndexType	ValueType	Change %	Baseline	Contender
COO	Int32Type	Int8	56.426	2459.397090	3847.138775
COO	Int16Type	Int8	43.062	2619.848028	3748.014603
COO	Int64Type	Int8	40.137	2850.826136	3995.057821
COO	Int8Type	Int8	35.478	2776.716694	3761.850287
COO	Int16Type	Int16	66.812	2394.011928	3993.489611
COO	Int32Type	Int16	59.281	2728.577337	4346.096797
COO	Int64Type	Int16	52.043	2596.404005	3947.663313
COO	Int8Type	Int16	44.510	2768.567992	4000.853968
COO	Int32Type	Float	93.283	2287.343291	4421.039474
COO	Int16Type	Float	71.100	2577.689569	4410.437858
COO	Int64Type	Float	61.035	2821.300996	4543.277655
COO	Int8Type	Float	53.303	2666.801559	4088.291362
COO	Int64Type	Double	88.230	2616.000738	4924.096764
COO	Int16Type	Double	70.188	2698.335495	4592.253023
COO	Int32Type	Double	69.368	2579.753902	4369.267270
COO	Int8Type	Double	62.781	2747.179806	4471.895533
CSR	Int8Type	Int8	-1.660	4626.181791	4549.389657
CSR	Int32Type	Int8	-4.718	4708.044556	4485.938002
CSR	Int64Type	Int8	-4.905	4687.961132	4458.024004
CSR	Int16Type	Int8	-7.164	4851.803479	4504.201161
CSR	Int32Type	Int16	-1.481	4722.676795	4652.711030
CSR	Int64Type	Int16	-5.853	4573.124314	4305.462839
CSR	Int8Type	Int16	-11.890	4631.290389	4080.628990
CSR	Int16Type	Int16	-12.372	4737.320600	4151.197839
CSR	Int64Type	Float	-4.999	4711.575565	4476.032868
CSR	Int16Type	Float	-5.062	4811.457037	4567.909440
CSR	Int32Type	Float	-7.651	4814.813084	4446.432341
CSR	Int8Type	Float	-12.196	5158.868330	4529.676101
CSR	Int32Type	Double	0.384	4631.005506	4648.789489
CSR	Int64Type	Double	-0.273	4816.669765	4803.522209
CSR	Int8Type	Double	-3.453	4646.652774	4486.217718
CSR	Int16Type	Double	-8.963	4952.032110	4508.161865
CSC	Int8Type	Int8	-4.207	4871.793338	4666.848429
CSC	Int64Type	Int8	-10.582	4571.251613	4087.509465
CSC	Int16Type	Int8	-13.189	4865.666295	4223.934008
CSC	Int32Type	Int8	-16.546	4999.577506	4172.360711
CSC	Int32Type	Int16	-0.619	4842.515715	4812.531287
CSC	Int8Type	Int16	-4.128	4689.737235	4496.135413
CSC	Int16Type	Int16	-8.110	4643.413441	4266.836346
CSC	Int64Type	Int16	-10.302	4971.879449	4459.696887
CSC	Int8Type	Float	-0.547	4694.347970	4668.667817
CSC	Int32Type	Float	-0.619	4718.207593	4688.994670
CSC	Int64Type	Float	-3.360	4847.753893	4684.849545
CSC	Int16Type	Float	-7.408	4956.795348	4589.607722
CSC	Int32Type	Double	0.491	4917.172760	4941.321519
CSC	Int64Type	Double	-0.754	5069.589290	5031.376333
CSC	Int16Type	Double	-2.023	4762.425910	4666.071367
CSC	Int8Type	Double	-5.987	4934.016268	4638.636384
CSF	Int16Type	Int8	25.394	7885.986312	9888.571309
CSF	Int64Type	Int8	22.605	9083.444559	11136.716151
CSF	Int32Type	Int8	13.486	8228.377064	9338.031561
CSF	Int8Type	Int8	11.900	7687.347967	8602.168534
CSF	Int64Type	Int16	28.094	8247.255772	10564.280045
CSF	Int8Type	Int16	20.276	8172.411486	9829.454135
CSF	Int32Type	Int16	15.240	8776.213727	10113.751856
CSF	Int16Type	Int16	13.707	8396.322326	9547.223214
CSF	Int16Type	Float	25.989	8340.996459	10508.758442
CSF	Int32Type	Float	20.562	8882.075299	10708.400993
CSF	Int8Type	Float	19.021	8507.160025	10125.299339
CSF	Int64Type	Float	1.769	9657.000323	9827.819671
CSF	Int8Type	Double	20.382	8380.183782	10088.211026
CSF	Int32Type	Double	14.353	9512.403696	10877.711426
CSF	Int16Type	Double	13.737	8536.269931	9708.870939
CSF	Int64Type	Double	12.413	9737.555217	10946.296736

[wesm]

@mrkn to improve the benchmark usefulness I would recommend increasing the size of the data being processed. I ran them locally and the COO benchmarks all run in less than 10 microseconds (some close to 1 microsecond) and at that speed things like destructors show up as non-trivial overhead

+ 91.29% 0.50% arrow-tensor-co arrow-tensor-conversion-benchmark [.] arrow::Int8RowMajorTensorConversionFixture_ConvertToSparseCOOTensorInt32_Benchmark::BenchmarkCase
+ 91.24% 0.00% arrow-tensor-co arrow-tensor-conversion-benchmark [.] _start
+ 91.24% 0.00% arrow-tensor-co libc-2.27.so [.] __libc_start_main
+ 91.24% 0.00% arrow-tensor-co arrow-tensor-conversion-benchmark [.] main
+ 91.24% 0.00% arrow-tensor-co arrow-tensor-conversion-benchmark [.] benchmark::RunSpecifiedBenchmarks
+ 79.50% 3.05% arrow-tensor-co arrow-tensor-conversion-benchmark [.] arrow::SparseTensorImpl<arrow::SparseCOOIndex>::Make
+ 74.47% 2.29% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::internal::MakeSparseCOOTensorFromTensor
+ 74.47% 0.04% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::internal::MakeSparseTensorFromTensor
+ 74.33% 16.65% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::internal::(anonymous namespace)::SparseCOOTensorConverter::Convert
+ 25.47% 25.47% arrow-tensor-co libc-2.27.so [.] __memmove_avx_unaligned_erms
+ 12.64% 7.00% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::internal::ComputeRowMajorStrides
+ 11.35% 1.87% arrow-tensor-co arrow-tensor-conversion-benchmark [.] arrow::SparseTensor::~SparseTensor
+ 9.15% 0.26% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::internal::IsTensorStridesContiguous
+ 7.76% 2.52% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::Tensor::CountNonZero
+ 7.25% 0.57% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::AllocateBuffer
+ 6.99% 6.99% arrow-tensor-co libc-2.27.so [.] cfree@GLIBC_2.2.5
+ 6.99% 0.00% arrow-tensor-co libc-2.27.so [.] __GI___libc_free (inlined)
+ 6.76% 1.07% arrow-tensor-co libstdc++.so.6.0.27 [.] operator new
+ 6.03% 0.00% arrow-tensor-co libc-2.27.so [.] __GI___libc_malloc (inlined)
+ 5.73% 1.71% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::PoolBuffer::~PoolBuffer
+ 5.42% 1.35% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::SparseCOOIndex::SparseCOOIndex
+ 4.99% 0.41% arrow-tensor-co libarrow.so.100.0.0 [.] arrow::SparseCOOIndex::~SparseCOOIndex

[wesm]

What do you think about pursuing the performance optimization work as a follow up?

[mrkn]

@wesm I decided to separate a pull-request for performance optimization because I may need some days to get this work done.
I'll make a new ticket for optimization, and clean up this pull-request.

[wesm]

OK, sounds good, let me know when this is ready to be merged

[wesm]

+1, CI is looking OK now. Thanks @mrkn!