ARROW-5111: [Go] implement reading list arrays from Arrow file

[sbinet]

No description provided.

[sbinet]

needs #4155

[codecov-io]

Codecov Report

Merging #4157 into master will decrease coverage by 27.23%.
The diff coverage is 0%.

@@ Coverage Diff @@## master #4157 +/- ##
===========================================
- Coverage 87.89% 60.65% -27.24% 
===========================================
Files 758 67 -691 Lines 92163 4883 -87280 Branches 1251 0 -1251 ===========================================
- Hits 81006 2962 -78044 + Misses 11040 1812 -9228 + Partials 117 109 -8

Impacted Files	Coverage Δ
go/arrow/ipc/file_reader.go	0% <0%> (ø)	⬆️
go/arrow/ipc/metadata.go	0% <0%> (ø)	⬆️
python/pyarrow/ipc.pxi
cpp/src/arrow/csv/chunker-test.cc
cpp/src/parquet/column_page.h
cpp/src/parquet/bloom_filter-test.cc
cpp/src/arrow/array/builder_decimal.cc
cpp/src/plasma/client.cc
cpp/src/arrow/io/test-common.h
cpp/src/arrow/util/int-util-test.cc
... and 683 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c0930d...dda12fd. Read the comment docs.

[nevi-me]

Hi @sbinet, I'm working on the equivalent for Rust. Hope you don't mind me asking a few questions.

• Does the binary array always consist of 3 buffers, or would it be 2 if the null count is 0? Here's the relevant code in the Rust impl: https://github.com/apache/arrow/pull/4167/files#diff-cd3519a5e748548b5323b0d05d8e9c8aR53 (cc @pitrou@kszucs )
• Do you mind sharing a script for generating Arrow data in Python (I took the one you used when you asked a question in the mailing list)? I haven't used pyarrow in a while, so I was struggling with making lists and structs.

Thanks

[pitrou]

A binary array always has 3 buffers, but the first buffer may be null or empty if the null count is 0.

This means buffer #1 should always be the offsets array and buffer #2 the values array.

[nevi-me]

Thanks @pitrou, is it safe for me to presume that the buffer numbers for primitives, list and struct will behave in the same way, where the null buffer may be null or empty?

[pitrou]

Yes, it's part of the Arrow format specification.

[sbinet]

@nevi-me here is what I am using:
https://gist.github.com/sbinet/8ec09eb95bc89c6d71cc192dfcce677b

[sbinet]

PTAL @stuartcarnie@alexandreyc@nickpoorman