improved support for struct columns with missing values

[baumgold]

The current implementation depends on Arrow.default returning a sentinel object when the value is missing. This usually works, but occasionally has problems. This PR simplifies the implementation and solves some of the problematic corner-cases.

One example of a problematic corner-case is a column of lists of structs:

julia> [(a=1,b=2), missing, (a=3,b=4)]
3-element Vector{Union{Missing, @NamedTuple{a::Int64, b::Int64}}}:
(a =1, b =2)
missing
(a =3, b =4)

It's possible that users may wrap this Vector in a SubArray. In this case Arrow.default(::Type{<:SubArray }) actually uses Arrow.default(::Type{<:AbstractVector}) which enforces that the parent-type is a Vector - this is not always the case. For example, the parent may well be of type Arrow.Struct if the data being written was also read from an Arrow file. Certainly we could enhance Arrow.default to return a proper type for SubArray but simpler is to remove the dependency on Arrow.default from ToStruct.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.36%. Comparing base (ac199b0) to head (78e22be).
⚠️ Report is 54 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #498 +/- ##
==========================================
+ Coverage 87.34% 87.36% +0.01% 
==========================================
Files 26 26 Lines 3288 3292 +4 ==========================================
+ Hits 2872 2876 +4 
Misses 416 416 

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[baumgold]

@quinnj / @ericphanson - Any questions/comments/concerns here? If not I'd like to merge and release. Thanks.

[ericphanson]

do we still want Base.nonmissingtype(eltype(A)) inside the field type?

[baumgold]

I don't think so. My change makes it so missing types are properly supported.

[ericphanson]

I don't really know this code well enough to be confident here. But one thing I can say is it's probably good if the PR adds a test that is failing on main, to show precisely what has been fixed (and prevent regressions)