GH-40428: [Python][CI] Fix dataset partition filter tests with pandas nightly

[jorisvandenbossche]

Rationale for this change

From debugging the failure, it seems this is due to pandas changing a filter operation to sometimes preserve a RangeIndex now instead of returning an Integer64Index. And the conversion to Arrow changes based on that (RangeIndex is metadata only by default, integer index becomes a column)

Therefore making the tests more robust to ensure there is always at least one non-partition column in the DataFrame, so it doesn't depend on the index whether the result is empty or not.

• GitHub Issue: [Python][CI] Dataset partition filter tests failing with pandas nightly #40428

[github-actions]

⚠️ GitHub issue #40428has been automatically assigned in GitHub to PR creator.

[jorisvandenbossche]

@github-actions crossbow submit test-conda-python-3.10-pandas-nightly

[github-actions]

Revision: 26c53c3

Submitted crossbow builds: ursacomputing/crossbow @ actions-3bc5d6c7a3

Task	Status
test-conda-python-3.10-pandas-nightly

[jorisvandenbossche]

@github-actions crossbow submit test-conda-python-3.10-pandas-nightly

[github-actions]

Revision: 62bf9dd

Submitted crossbow builds: ursacomputing/crossbow @ actions-a15ea740b7

Task	Status
test-conda-python-3.10-pandas-nightly

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 788200a.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.