Skip to content

[Python][Interchange] from_dataframe drops the categorical ordered flag #51046

Description

@Kayvan-Zahiri

Describe the bug, including details regarding any error messages, version, and platform.

pyarrow.interchange.from_dataframe builds the dictionary array without the is_ordered flag the producer reports, so an ordered categorical column comes back unordered. A pyarrow table whose column is dictionary(int32, string, ordered=True) does not survive its own round trip, and an ordered pandas Categorical loses its ordering on the way in.

importpyarrowaspafrompyarrow.interchangeimportfrom_dataframet=pa.table({"c": pa.array(["a", "b", "a"]).dictionary_encode().cast(
pa.dictionary(pa.int32(), pa.string(), ordered=True))})
print(t.schema.field("c").type.ordered) # Trueprint(from_dataframe(t).schema.field("c").type.ordered) # False

The interchange protocol exposes the flag: a categorical column's describe_categorical returns is_ordered, and the pandas consumer passes it through. The pyarrow consumer reads the mapping but drops is_ordered.

Component(s)

Python

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions