Some libraries (e.g. NumPy) represent boolean values using an array of int8 or uint8 values of 1's and 0's. This can present a challenge at times to receive such memory without copying.
Now that we have ExtensionType capabilities, we could define an extension type distinguish UInt8/Int8-annotated-as-boolean to be able to flow through such data in applications.
A discussion about introducing a new logical type didn't go anywhere, so having a custom container that can be used for these specialized applications is one way to unblock the use case. If we develop some endogenous use of such data in C++, we would need to be mindful to sanitize it to bitpacked boolean before sending to another Arrow application
Reporter: Wes McKinney / @wesm
PRs and other links:
Note: This issue was originally created as ARROW-1674. Please see the migration documentation for further details.
Some libraries (e.g. NumPy) represent boolean values using an array of int8 or uint8 values of 1's and 0's. This can present a challenge at times to receive such memory without copying.
Now that we have ExtensionType capabilities, we could define an extension type distinguish UInt8/Int8-annotated-as-boolean to be able to flow through such data in applications.
A discussion about introducing a new logical type didn't go anywhere, so having a custom container that can be used for these specialized applications is one way to unblock the use case. If we develop some endogenous use of such data in C++, we would need to be mindful to sanitize it to bitpacked boolean before sending to another Arrow application
Reporter: Wes McKinney / @wesm
PRs and other links:
Note: This issue was originally created as ARROW-1674. Please see the migration documentation for further details.