ARROW-2104: [C++] take kernel functions for nested types

[bkietz]

Take now supports gathering from List, FixedSizeList, Map, and Struct arrays. Union is not yet supported

[pitrou]

The implementation is neat! Some comments below.

[pitrou]

Hmm... it would probably be more efficient to do it in two passes:

• first pass to compute the overall data length
• second pass to do the actual builder appends after having reserved the right data size at once

What do you think? Is it easily doable to switch to such a scheme?

[bkietz]

That's not a problem. I'll write a benchmark so we can watch the difference.

[bkietz]

@pitrou as it turns out, it's actually a little faster to ReserveData for each string:

# ReserveData on append
---------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
---------------------------------------------------------------------------------------------
FilterString/1048576/1/min_time:1.000 1586715 ns 1586705 ns 884 null_percent=1 size=1048.58k 630.237MB/s
FilterString/8388608/1/min_time:1.000 23555110 ns 23555006 ns 72 null_percent=1 size=8.38861M 339.631MB/s
# separate ReserveData pass
---------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
---------------------------------------------------------------------------------------------
FilterString/1048576/1/min_time:1.000 1656093 ns 1654443 ns 841 null_percent=1 size=1048.58k 604.433MB/s
FilterString/8388608/1/min_time:1.000 19849301 ns 19809235 ns 71 null_percent=1 size=8.38861M 403.852MB/s

Would you like me to push this change so you can inspect it?

[pitrou]

It's faster on the 1MB case, but slower on the 8MB case. I expect it to become even slower on larger data. Can you try with e.g. 32MB?

[pitrou]

(what is the size of your L3 cache?)

[pitrou]

Update: we discussed and investigated it a bit with @bkietz . It turns out that the dominating contributor to Filter performance is probably reading the selection bitmap, so iterating twice on the input instead of once has a negative impact.

[bkietz]

The benchmarks for Filter with this refactor show a slight improvement over previous:

----------------------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
----------------------------------------------------------------------------------------------------------
FilterInt64/32768/0/min_time:1.000 16321 ns 16321 ns 85904 null_percent=0 size=32.768k 1.86981GB/s
FilterInt64/32768/1/min_time:1.000 16431 ns 16431 ns 84487 null_percent=1 size=32.768k 1.85729GB/s
FilterInt64/32768/10/min_time:1.000 19935 ns 19935 ns 70396 null_percent=10 size=32.768k 1.53088GB/s
FilterInt64/32768/50/min_time:1.000 39153 ns 39153 ns 35976 null_percent=50 size=32.768k 798.148MB/s
FilterInt64/1048576/1/min_time:1.000 882180 ns 882176 ns 1590 null_percent=1 size=1048.58k 1.10699GB/s
FilterInt64/8388608/1/min_time:1.000 7038037 ns 7038002 ns 200 null_percent=1 size=8.38861M 1.11005GB/s
FilterFixedSizeList1Int64/32768/0/min_time:1.000 44414 ns 44414 ns 31993 null_percent=0 size=32.768k 703.614MB/s
FilterFixedSizeList1Int64/32768/1/min_time:1.000 45416 ns 45416 ns 30649 null_percent=1 size=32.768k 688.08MB/s
FilterFixedSizeList1Int64/32768/10/min_time:1.000 57006 ns 57006 ns 24561 null_percent=10 size=32.768k 548.193MB/s
FilterFixedSizeList1Int64/32768/50/min_time:1.000 104088 ns 104088 ns 13318 null_percent=50 size=32.768k 300.227MB/s
FilterFixedSizeList1Int64/1048576/1/min_time:1.000 1590805 ns 1590798 ns 887 null_percent=1 size=1048.58k 628.615MB/s
FilterFixedSizeList1Int64/8388608/1/min_time:1.000 12718793 ns 12718723 ns 110 null_percent=1 size=8.38861M 628.994MB/s

[pitrou]

It doesn't seem right to divide by int64_t here.

[bkietz]

@fsaintjacques could you comment?

[fsaintjacques]

My intent was to make the benchmark share a memory budget (args.size) and let them size the input. The goal is to be able to compare operations. So if you don't input any physical sizes and just input length, no need to divide.

[pitrou]

Right. So the problem here is to generate a random string array roughly consistent with the given memory budget.

[bkietz]

CI failures seem spurious:

• Travis pyarrow test_jvm_types
• Travis unknown type name '__m128i'
• Appveyor OpenSSL build error

[pitrou]

Rebased to fix unrelated CI failures.

[bkietz]

@pitrou when you rebased you missed the last two commits somehow ( a4bfaac3fe5e63569d5e70864d19ae59b25f5ae1 9cee25e787fea043a0e749809d267805429f199f ).

I'll update the filter benchmark to produce a StringArray with ~(memory budget) total characters and rebase again.

[pitrou]

@bkietz Wow, not sure how that happened :-/ Sorry for that.

[bkietz]

No worries, easily fixed

[bkietz]

Spurious Appveyor failure while downloading R installer

[wesm]

There was an idiosyncratic Maven failure. I restarted the one failing Travis job

[wesm]

Appveyor failed due to the ongoing cloud.r-project.org DNS issue. We should probably add this test to "allow_failures" until that is resolved

[bkietz]

@wesm anything I need to do to accomplish that?

[ursabot]

AMD64 Ubuntu 18.04 C++ Benchmark (#27346) builder has been succeeded.

Revision: 73262bd

 =================================== =========== =========== ==========
benchmark baseline contender change
=================================== =========== =========== ==========
FilterString/32768/1/min_time:1.000 4.33183e+09 4.25759e+09 -0.0171373
=================================== =========== =========== ==========

[fsaintjacques]

@ursabot benchmark --benchmark-filter=TakeString HEAD^1

[ursabot]

AMD64 Ubuntu 18.04 C++ Benchmark (#27347) builder failed.

Revision: 73262bd

Archery: 'archery benchmark ...' step's stderr:

Selected compiler gcc 7.4.0
Using ld linker
Configured for RELEASE build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})
CMake Warning at cmake_modules/ThirdpartyToolchain.cmake:176 (find_package):
No "Findbenchmark.cmake" found in CMAKE_MODULE_PATH.
Call Stack (most recent call first):
cmake_modules/ThirdpartyToolchain.cmake:1610 (resolve_dependency)
CMakeLists.txt:389 (include)
CMake Warning (dev) at cmake_modules/ThirdpartyToolchain.cmake:176 (find_package):
Findbenchmark.cmake must either be part of this project itself, in this
case adjust CMAKE_MODULE_PATH so that it points to the correct location
inside its source tree.
Or it must be installed by a package which has already been found via
find_package(). In this case make sure that package has indeed been found
and adjust CMAKE_MODULE_PATH to contain the location where that package has
installed Findbenchmark.cmake. This must be a location provided by that
package. This error in general means that the buildsystem of this project
is relying on a Find-module without ensuring that it is actually available.
Call Stack (most recent call first):
cmake_modules/ThirdpartyToolchain.cmake:1610 (resolve_dependency)
CMakeLists.txt:389 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
Cloning into '/tmp/arrow-bench-t9n1i_r7/'HEAD^1'/arrow'...
done.
error: pathspec ''HEAD^1'' did not match any file(s) known to git.
Traceback (most recent call last):
File "/usr/local/bin/archery", line 11, in <module>
load_entry_point('archery', 'console_scripts', 'archery')()
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/buildbot/AMD64_Ubuntu_18_04_C___Benchmark/dev/archery/archery/cli.py", line 384, in benchmark_diff
suite_filter=suite_filter, benchmark_filter=benchmark_filter)
File "/buildbot/AMD64_Ubuntu_18_04_C___Benchmark/dev/archery/archery/benchmark/runner.py", line 75, in from_rev_or_path
src_rev, _ = src.at_revision(rev_or_path, clone_dir)
File "/buildbot/AMD64_Ubuntu_18_04_C___Benchmark/dev/archery/archery/utils/source.py", line 90, in at_revision
git.checkout(revision, git_dir=clone_dir)
File "/buildbot/AMD64_Ubuntu_18_04_C___Benchmark/dev/archery/archery/utils/git.py", line 30, in wrapper
return fn(self, sub_cmd, *argv, **kwargs)
File "/buildbot/AMD64_Ubuntu_18_04_C___Benchmark/dev/archery/archery/utils/git.py", line 52, in checkout
return self.run_cmd(*argv, **kwargs)
File "/buildbot/AMD64_Ubuntu_18_04_C___Benchmark/dev/archery/archery/utils/git.py", line 44, in run_cmd
return self.run(*opts, cmd, *argv, **kwargs)
File "/buildbot/AMD64_Ubuntu_18_04_C___Benchmark/dev/archery/archery/utils/command.py", line 68, in run
return subprocess.run(invocation, **kwargs)
File "/usr/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/git', '-C', "/tmp/arrow-bench-t9n1i_r7/'HEAD^1'/arrow", 'checkout', "'HEAD^1'"]' returned non-zero exit status 1.

[fsaintjacques]

@ursabot benchmark --help

[ursabot]

Usage: @ursabot benchmark [OPTIONS] [BASELINE]
Trigger all benchmarks registered for this pull request.
Options:
--suite-filter <regex> Regex filtering benchmark suites.
--benchmark-filter <regex> Regex filtering benchmarks.
--help Show this message and exit.

[fsaintjacques]

@ursabot benchmark --benchmark-filter=TakeString HEAD~2

[ursabot]

AMD64 Ubuntu 18.04 C++ Benchmark (#27353) builder failed with an exception.

Revision: 73262bd

Archery: 'archery benchmark ...' step's traceback:

Traceback (most recent call last):
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/internet/defer.py", line 1475, in gotResult
_inlineCallbacks(r, g, status)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
--- <exception caught here> ---
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/buildbot/process/buildstep.py", line 566, in startStep
self.results = yield self.run()
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "/home/ursabot/ursabot/ursabot/steps.py", line 43, in run
await log.addContent(content)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/buildbot/process/log.py", line 130, in addContent
return self.lbf.append(text)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/buildbot/util/lineboundaries.py", line 62, in append
text = self.newline_re.sub('\n', text)
builtins.TypeError: expected string or bytes-like object

[fsaintjacques]

@ursabot benchmark --benchmark-filter=TakeString HEAD~1

[ursabot]

AMD64 Ubuntu 18.04 C++ Benchmark (#27354) builder failed with an exception.

Revision: 73262bd

Archery: 'archery benchmark ...' step's traceback:

Traceback (most recent call last):
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/internet/defer.py", line 1475, in gotResult
_inlineCallbacks(r, g, status)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
--- <exception caught here> ---
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/buildbot/process/buildstep.py", line 566, in startStep
self.results = yield self.run()
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "/home/ursabot/ursabot/ursabot/steps.py", line 43, in run
await log.addContent(content)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/buildbot/process/log.py", line 130, in addContent
return self.lbf.append(text)
File "/home/ursabot/.conda/envs/ursabot/lib/python3.7/site-packages/buildbot/util/lineboundaries.py", line 62, in append
text = self.newline_re.sub('\n', text)
builtins.TypeError: expected string or bytes-like object

[fsaintjacques]

@ursabot benchmark --benchmark-filter=TakeString eaf8302

[fsaintjacques]

@ursabot benchmark --help

[ursabot]

Usage: @ursabot benchmark [OPTIONS] [BASELINE]
Trigger all benchmarks registered for this pull request.
Options:
--suite-filter <regex> Regex filtering benchmark suites.
--benchmark-filter <regex> Regex filtering benchmarks.
--help Show this message and exit.

[fsaintjacques]

@ursabot benchmark --help

[ursabot]

Usage: @ursabot benchmark [OPTIONS] [<baseline>]
Run the benchmark suite in comparison mode.
This command will run the benchmark suite for tip of the branch commit
against `<baseline>` (or master if not provided).
Examples:
# Run the all the benchmarks
�
@ursabot benchmark
# Compare only benchmarks where the name matches the `^Sum` regex
�
@ursabot benchmark --benchmark-filter=^Sum
# Compare only benchmarks where the suite matches the `compute-` regex.
# A suite is the C++ binary.
�
@ursabot benchmark --suite-filter=compute-
# Sometimes a new optimization requires the addition of new benchmarks to
# quantify the performance increase. When doing this be sure to add the
# benchmark in a separate commit before introducing the optimization.
#
# Note that specifying the baseline is the only way to compare a new
# benchmark, otherwise the intersection of benchmarks with master will be
# empty (no comparison possible).
#
# Suppose that the "MyBenchmark" benchmark is introduced in HEAD~2 and the
# optimization is found in HEAD. The following command will show the
# difference with and without the optimization on said benchmark.
�
@ursabot benchmark --benchmark-filter=MyBenchmark HEAD~2
Options:
--suite-filter <regex> Regex filtering benchmark suites.
--benchmark-filter <regex> Regex filtering benchmarks.
--help Show this message and exit.

[fsaintjacques]

@ursabot benchmark --help

[ursabot]

Usage: @ursabot benchmark [OPTIONS] [<baseline>]
Run the benchmark suite in comparison mode.
This command will run the benchmark suite for tip of the branch commit
against `<baseline>` (or master if not provided).
Examples:
# Run the all the benchmarks
@ursabot benchmark
# Compare only benchmarks where the name matches the /^Sum/ regex
@ursabot benchmark --benchmark-filter=^Sum
# Compare only benchmarks where the suite matches the /compute-/ regex.
# A suite is the C++ binary.
@ursabot benchmark --suite-filter=compute-
# Sometimes a new optimization requires the addition of new benchmarks to
# quantify the performance increase. When doing this be sure to add the
# benchmark in a separate commit before introducing the optimization.
#
# Note that specifying the baseline is the only way to compare a new
# benchmark, otherwise the intersection of benchmarks with master will be
# empty (no comparison possible).
#
# The following command compares the results of matching benchmarks,
# compiling against HEAD and the provided baseline commit (HEAD~2). You can
# use this to quantify the performance improvement of new optimizations or
# to check for regressions.
@ursabot benchmark --benchmark-filter=MyBenchmark HEAD~2
Options:
--suite-filter <regex> Regex filtering benchmark suites.
--benchmark-filter <regex> Regex filtering benchmarks.
--help Show this message and exit.

[fsaintjacques]

@ursabot benchmark --benchmark-filter=TakeString eaf8302

[ursabot]

AMD64 Ubuntu 18.04 C++ Benchmark (#27385) builder has been succeeded.

Revision: 73262bd

 =================================== =========== =========== ===========
benchmark baseline contender change
=================================== =========== =========== ===========
TakeString/32768/50/min_time:1.000 1.33361e+09 1.27811e+09 -0.0416122
TakeString/32768/1/min_time:1.000 1.54006e+09 1.54192e+09 0.00120504
TakeString/8388608/1/min_time:1.000 7.39268e+08 8.28827e+08 0.121145
TakeString/32768/0/min_time:1.000 1.74939e+09 1.73941e+09 -0.00570733
TakeString/32768/10/min_time:1.000 1.52927e+09 1.51239e+09 -0.0110394
TakeString/1048576/1/min_time:1.000 1.37075e+09 1.36925e+09 -0.00109342
=================================== =========== =========== ===========

[bkietz]

@wesm@pitrou the only remaining CI failure is an abort in test_flight.py

[bkietz]

Follow up JIRA for optimization of these kernels:https://issues.apache.org/jira/browse/ARROW-5760

[wesm]

can we fix the hideous benchmark formatting to show human readable timings?

[bkietz]

Not with a change to Take(), I think

[wesm]

Sorry that was a general comment to @fsaintjacques =)

I restarted the failed Travis build to see if it's transient

https://travis-ci.org/apache/arrow/jobs/551366263

definitely concerning that Flight is able to crash like that

[fsaintjacques]

@wesm actively working to this next. See voltrondata-labs/ursabot#98

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@7ae6a58). Click here to learn what that means.
The diff coverage is 96.49%.

@@ Coverage Diff @@## master #4531 +/- ##
=========================================
Coverage ? 88.87% =========================================
Files ? 717 Lines ? 98750 Branches ? 0 =========================================
Hits ? 87768 Misses ? 10982 Partials ? 0

Impacted Files	Coverage Δ
cpp/src/arrow/compute/kernels/filter.h	66.66% <ø> (ø)
cpp/src/arrow/array.h	98.29% <ø> (ø)
cpp/src/arrow/compute/kernels/util-internal.h	100% <ø> (ø)
cpp/src/arrow/compute/kernels/take.cc	97.91% <100%> (ø)
cpp/src/arrow/buffer-builder.h	100% <100%> (ø)
cpp/src/arrow/compute/kernels/take-test.cc	100% <100%> (ø)
cpp/src/arrow/compute/kernels/filter.cc	98.03% <100%> (ø)
cpp/src/arrow/array/builder_primitive.cc	92.53% <100%> (ø)
cpp/src/arrow/compute/kernels/take.h	75% <100%> (ø)
cpp/src/arrow/array.cc	80.33% <100%> (ø)
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7ae6a58...73262bd. Read the comment docs.

[wesm]

+1