[CI][Python] Pandas nightly tests failing due to DatetimeIndex missmatch

[AlenkaF]

Describe the bug, including details regarding any error messages, version, and platform.

Nightly pandas jobs are failing due to AssertionError in pyarrow/tests/test_pandas.py:

 def test_table_column_subset_metadata(self):
 # ARROW-1883
 # non-default index
 for index in [
 pd.Index(['a', 'b', 'c'], name='index'),
 pd.date_range("2017-01-01", periods=3, tz='Europe/Brussels')]:
 df = pd.DataFrame({'a': [1, 2, 3],
 'b': [.1, .2, .3]}, index=index)
 table = pa.Table.from_pandas(df)
 
 table_subset = table.remove_column(1)
 result = table_subset.to_pandas()
 expected = df[['a']]
 if isinstance(df.index, pd.DatetimeIndex):
 df.index.freq = None
> tm.assert_frame_equal(result, expected)
E AssertionError: (None, <Day>)

See: https://github.com/ursacomputing/crossbow/actions/runs/25352255483/job/74334226712#step:6:4296

Component(s)

Continuous Integration, Python

[AnkitAhlawat7742]

Hi @AlenkaF ,
So, this failure already existed in the test case, and the PR #49878
simply exposed a pre-existing bug in the test. is it ?

[AlenkaF]

Yes, it seems so. The failure has been present before and is not connected to your PR.

[AnkitAhlawat7742]

Hmm… I checked the test failure and found that the test was incorrectly setting df.index.freq = None instead of expected.index.freq = None. I’ll open a separate PR to fix this once #49878 is merged. Also, thanks for opening issue #49920!

[AlenkaF]

I think there has been a change in Pandas that triggered this failure. From a quick search I would probably think it is this PR: pandas-dev/pandas#65429.

So what would be needed is to understand what was changed on the pandas side and then propose a solution in our test that is failing.

[AnkitAhlawat7742]

Thanks for sharing the link ,
After investigating pandas-dev/pandas#65429, the issue is clear — since freq is now Index-only state, expected = df[["a"]] creates an independent DatetimeIndex with its own freq=<Day>. Setting df.index.freq = None has no effect on expected.index. The fix is to set expected.index.freq = None instead. I have tested this locally and it passes.

[jorisvandenbossche]

Duplicate of #49933