ARROW-11299: [Python] Fix invalid-offsetof warnings

[cyb70289]

Use unique_ptr to hold FunctionOptions derived classes instances to fix
invalid-offsetof python build warnings.

~/arrow/python/build/temp.linux-x86_64-3.6/_compute.cpp:26034:146:
warning: offsetof within non-standard-layout type ‘__pyx_obj_7pyarrow_8_compute__CastOptions’ is undefined [-Winvalid-offsetof]
x_type_7pyarrow_8_compute__CastOptions.tp_weaklistoffset = offsetof(struct __pyx_obj_7pyarrow_8_compute__CastOptions, __pyx_base.__pyx_base.weakref);

[github-actions]

https://issues.apache.org/jira/browse/ARROW-11299

[cyb70289]

All debug build failed due to this change. FunctionOptions is supposed to be polymorphic, but looks not necessary.
Would like to hear comments before moving on.
@bkietz@pitrou

[bkietz]

@cyb70289arrow::dataset::Expression::Call::options requires that virtual destructor since a shared_ptr<FunctionOptions> is used to store subclasses of FunctionOptions. Removing this virtual destructor results in a resource leak since the derived classes' (implicit) destructors are not being called, leaking data members such as arrow::compute::SetLookupOptions::value_set which are not destroyed.

[bkietz]

I think the best way to resolve this issue is to avoid direct instances of derived classes of FunctionOptions in Cython, since Cython inappropriately applies offsetof. For example, note that the warning is emitted for MinMaxOptions but not for SetLookupOptions (where the options are held by unique_ptr rather than being a direct member).

I'd recommend the following approach: rewrite _compute.pyx::FunctionOptions to hold a shared_ptr[CFunctionOptions], and let the cdef classes acquire a non owning pointer to the appropriate FunctionOptions subclass. For example:

_compute.pxd

cdefclassFunctionOptions(_Weakrefable):
cdef:
shared_ptr[CFunctionOptions] optionscdefconstCFunctionOptions*get_options(self) exceptNULL

_compute.pyx

cdefclass_CastOptions(FunctionOptions):
cdef:
constCCastOptions*cast_options__slots__= () # avoid mistakingly creating attributescdefconstCFunctionOptions*get_options(self) exceptNULL:
returnself.cast_options

[cyb70289]

@cyb70289arrow::dataset::Expression::Call::options requires that virtual destructor since a shared_ptr<FunctionOptions> is used to store subclasses of FunctionOptions. Removing this virtual destructor results in a resource leak since the derived classes' (implicit) destructors are not being called, leaking data members such as arrow::compute::SetLookupOptions::value_set which are not destroyed.

Thanks @bkietz , I missed this point.
Will update according to your comment.

[bkietz]

LGTM, thanks!