[C++] Reduce shared library / binary code size (umbrella issue)

[asfimport]

We're reaching a point where we may need to be careful about decisions that increase code size:

• Instantiating too many templates for code that isn't performance sensitive, or where some templates may do the same thing (e.g. Int32Type kernels may do the same thing as a Date32Type kernel)

• Inlining functions that don't need to be inline

  Code size tends to correlate also with compilation times, but not always.

  I'll use this umbrella issue to organize issues related to reducing compiled code size

  At this moment (2020-05-27), here are the 25 largest object files in a -O2 build

  524896	src/arrow/CMakeFiles/arrow_objlib.dir/array/builder_dict.cc.o
  531920	src/arrow/CMakeFiles/arrow_objlib.dir/filesystem/s3fs.cc.o
  552000	src/arrow/CMakeFiles/arrow_objlib.dir/json/converter.cc.o
  575920	src/arrow/CMakeFiles/arrow_objlib.dir/csv/converter.cc.o
  595112	src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_cast_string.cc.o
  645728	src/arrow/CMakeFiles/arrow_objlib.dir/type.cc.o
  683040	src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_set_lookup.cc.o
  702232	src/arrow/CMakeFiles/arrow_objlib.dir/ipc/reader.cc.o
  729912	src/arrow/CMakeFiles/arrow_objlib.dir/tensor/coo_converter.cc.o
  752776	src/arrow/CMakeFiles/arrow_objlib.dir/tensor/csc_converter.cc.o
  752776	src/arrow/CMakeFiles/arrow_objlib.dir/tensor/csr_converter.cc.o
  877680	src/arrow/CMakeFiles/arrow_objlib.dir/array/dict_internal.cc.o
  885624	src/arrow/CMakeFiles/arrow_objlib.dir/builder.cc.o
  919072	src/arrow/CMakeFiles/arrow_objlib.dir/scalar.cc.o
  941776	src/arrow/CMakeFiles/arrow_objlib.dir/ipc/json_internal.cc.o
  1055248	src/arrow/CMakeFiles/arrow_objlib.dir/ipc/json_simple.cc.o
  1233304	src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_compare.cc.o
  1265160	src/arrow/CMakeFiles/arrow_objlib.dir/sparse_tensor.cc.o
  1343480	src/arrow/CMakeFiles/arrow_objlib.dir/tensor/csf_converter.cc.o
  1346928	src/arrow/CMakeFiles/arrow_objlib.dir/array.cc.o
  1502568	src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/vector_hash.cc.o
  1609760	src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_cast_numeric.cc.o
  1794416	src/arrow/CMakeFiles/arrow_objlib.dir/array/diff.cc.o
  2759552	src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/vector_filter.cc.o
  7609432	src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/vector_take.cc.o

Reporter: Wes McKinney / @wesm
Assignee: Wes McKinney / @wesm

Related issues:

• [C++] Reducing the code size of the tensor module (relates to)
• [C++] Reduce number of take kernels (relates to)
• [C++] Reduce generated code in vector_hash.cc (relates to)
• [C++] Reduce generated code in compute/kernels/scalar_compare.cc (relates to)
• [C++] Optimize Filter implementation (relates to)
• [C++] diff.cc is extremely slow to compile (relates to)
• [C++] Refine TransferBitmap template parameters (relates to)
• [C++] BitUtil::SetBitsTo probably doesn't need to be inline (is related to)

_{Note: This issue was originally created as ARROW-8970. Please see the migration documentation for further details.}

[asfimport]

Wes McKinney / @wesm:
After ARROW-7784, ARROW-5760, and ARROW-9075 patches libarrow.so is now down to 18.44 MB from 23.09 MB in -O3 build on clang-8

Now here are the largest object files in the build

$ find src -type f -printf '%s %p\n' | sort -nr | head -20
1421728 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_cast_numeric.cc.o
1284672 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_compare.cc.o
1203344 src/arrow/CMakeFiles/arrow_objlib.dir/sparse_tensor.cc.o
1145640 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/vector_hash.cc.o
905088 src/arrow/CMakeFiles/arrow_objlib.dir/scalar.cc.o
828072 src/arrow/CMakeFiles/arrow_objlib.dir/ipc/json_simple.cc.o
811544 src/arrow/CMakeFiles/arrow_objlib.dir/tensor/csf_converter.cc.o
727448 src/arrow/CMakeFiles/arrow_objlib.dir/ipc/json_internal.cc.o
676576 src/arrow/CMakeFiles/arrow_objlib.dir/array/array_dict.cc.o
668904 src/arrow/CMakeFiles/arrow_objlib.dir/type.cc.o
632680 src/arrow/CMakeFiles/arrow_objlib.dir/array/array_base.cc.o
619968 src/arrow/CMakeFiles/arrow_objlib.dir/builder.cc.o
617392 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/vector_selection.cc.o
583160 src/arrow/CMakeFiles/arrow_objlib.dir/tensor/csr_converter.cc.o
583160 src/arrow/CMakeFiles/arrow_objlib.dir/tensor/csc_converter.cc.o
554792 src/arrow/CMakeFiles/arrow_objlib.dir/ipc/reader.cc.o
554144 src/arrow/CMakeFiles/arrow_objlib.dir/tensor/coo_converter.cc.o
540912 src/arrow/CMakeFiles/arrow_objlib.dir/array/util.cc.o
500088 src/arrow/CMakeFiles/arrow_objlib.dir/array/diff.cc.o
473096 src/arrow/CMakeFiles/arrow_objlib.dir/filesystem/s3fs.cc.o

[asfimport]

Wes McKinney / @wesm:
@rok @mrkn the tensor modules have quite a lot of object code, do you think there's anything that can be done to streamline some of these functions to produce smaller code?

[asfimport]

Wes McKinney / @wesm:
@pitrou what do you think about moving the ipc/json_* object code to libarrow_testing?

[asfimport]

Uwe Korn / @xhochy:
@wesm Code size can often be reduced by compiling with link time optimization as this uncovers duplicated code paths and merges them, might be an easy win if working.

[asfimport]

Wes McKinney / @wesm:
I looked at some of the large files and LTO won't help them in many cases since many of the inline functions that are generating the code bloat are only used in those files.

[asfimport]

Antoine Pitrou / @pitrou:
No strong feelings about ipc/json*, but I'm frankly not sure if that's worth the hassle.
It would be nice if one could build integration tests without all the other tests, though.

[asfimport]

Wes McKinney / @wesm:
@pitrou I think that's best handled through cmake targets, e.g. ninja integration

[asfimport]

Kenta Murata / @mrkn:
@wesm There are some options to reduce code sizes related to tensor modules.

(1) Reducing inline functions.
(2) Reducing sparse tensor's index value types. I guess we needn't support 8-bit and 16-bit types for sparse tensor index. At least scipy's sparse matrix doesn't use 8bit and 16bit types for sparse matrix indices. We should investigate other libraries providing sparse tensors.
(3) Dropping functions for converting among dense and sparse tensors. These functions aren't necessary to exchanging existing tensor data between systems.

I will start (1) and (2) right now. If we don't need to provide conversion functions, I will also start (3).

[asfimport]

Wes McKinney / @wesm:
For those who might be interested, here are currently the 100 largest symbols in libarrow.so:

https://gist.github.com/wesm/98307fa12e05760921d8f15e30291920

I was a bit shocked to learn that see that our use of Brotli causes a bunch of data to be statically linked into the .so. But in general it seems that streamlining the Tensor code would bring a significant reduction in binary size

[asfimport]

Wes McKinney / @wesm:
We've made progress – with my set of enabled components libarrow.so is down to 13.3 MB from 22.0 MB in 0.17.0

$ ll release/
total 36256
lrwxrwxrwx 1 wesm wesm 14 Jul 5 15:16 libarrow.so -> libarrow.so.17*
lrwxrwxrwx 1 wesm wesm 19 Jul 5 10:43 libarrow.so.100 -> libarrow.so.100.0.0*
-rwx------ 1 wesm wesm 13986464 Jul 5 10:43 libarrow.so.100.0.0*
lrwxrwxrwx 1 wesm wesm 18 Jul 5 15:16 libarrow.so.17 -> libarrow.so.17.0.0*
-rwx------ 1 wesm wesm 23134296 Jul 5 15:16 libarrow.so.17.0.0*

Here are the largest object files now

$ find src/arrow/CMakeFiles/arrow_objlib.dir/ -printf '%s %p\n'| sort -nr | head -30
806128 src/arrow/CMakeFiles/arrow_objlib.dir/ipc/json_simple.cc.o
771784 src/arrow/CMakeFiles/arrow_objlib.dir/scalar.cc.o
675768 src/arrow/CMakeFiles/arrow_objlib.dir/array/array_dict.cc.o
671360 src/arrow/CMakeFiles/arrow_objlib.dir/type.cc.o
625720 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/vector_selection.cc.o
624920 src/arrow/CMakeFiles/arrow_objlib.dir/builder.cc.o
589664 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/vector_hash.cc.o
562312 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_compare.cc.o
559296 src/arrow/CMakeFiles/arrow_objlib.dir/ipc/reader.cc.o
547392 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_cast_numeric.cc.o
500344 src/arrow/CMakeFiles/arrow_objlib.dir/array/diff.cc.o
484360 src/arrow/CMakeFiles/arrow_objlib.dir/array/array_base.cc.o
437080 src/arrow/CMakeFiles/arrow_objlib.dir/json/converter.cc.o
401816 src/arrow/CMakeFiles/arrow_objlib.dir/csv/converter.cc.o
380152 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/vector_sort.cc.o
375536 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/aggregate_basic.cc.o
363912 src/arrow/CMakeFiles/arrow_objlib.dir/array/builder_dict.cc.o
355656 src/arrow/CMakeFiles/arrow_objlib.dir/array/util.cc.o
343752 src/arrow/CMakeFiles/arrow_objlib.dir/pretty_print.cc.o
337936 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_cast_internal.cc.o
309976 src/arrow/CMakeFiles/arrow_objlib.dir/json/parser.cc.o
284272 src/arrow/CMakeFiles/arrow_objlib.dir/ipc/writer.cc.o
283128 src/arrow/CMakeFiles/arrow_objlib.dir/array/array_nested.cc.o
261264 src/arrow/CMakeFiles/arrow_objlib.dir/c/bridge.cc.o
260632 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_set_lookup.cc.o
260312 src/arrow/CMakeFiles/arrow_objlib.dir/csv/reader.cc.o
259032 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_cast_string.cc.o
226528 src/arrow/CMakeFiles/arrow_objlib.dir/io/file.cc.o
226008 src/arrow/CMakeFiles/arrow_objlib.dir/ipc/metadata_internal.cc.o
210928 src/arrow/CMakeFiles/arrow_objlib.dir/compute/kernels/scalar_string.cc.o

There are a number of issues with the largest symbols that bear closer examination

https://gist.github.com/wesm/6a0dc08cbb26d5a3c125bdd416bb5d04

for example Scalar::CastTo is taking up 75K alone. All the arrow_vendored::date formatting stuff is weird (20K to format a date?), too.

[asfimport]

Antoine Pitrou / @pitrou:
How did you find out that Scalar::CastTo is taking 75K alone? Running the nm --print-size --size-sort line I couldn't see it?

Generally, I think we're currently fine in terms of code size generated, given the breadth of features.

[asfimport]

Wes McKinney / @wesm:
Since we've made a lot of progress since 0.17.0 I'm declaring this done for now. We can pursue further binary trimming work in new issues