GH-45129: [Python][C++] Fix usage of deprecated C++ functionality on pyarrow

[raulcd]

Rationale for this change

We are using two C++ deprecated APIs:

• we are using decimal instead of smallest_decimal
• we are using Arrow:Status on GetRecordBatchReader instead of Arrow::Result

What changes are included in this PR?

Update code to use non deprecated functions.

Are these changes tested?

Yes via CI with existing tests.

Are there any user-facing changes?

No

• GitHub Issue: [Python][C++] Fix deprecations when building pyarrow #45129

[github-actions]

⚠️ GitHub issue #45129has been automatically assigned in GitHub to PR creator.

[raulcd]

@github-actions crossbow submit -g python

[github-actions]

Revision: c1deb19cdb624e1f5e8363fc73c6afe28529270f

Submitted crossbow builds: ursacomputing/crossbow @ actions-d40040ac1a

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-cython2
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest-numpy-latest
test-conda-python-3.10-substrait
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-1.26
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-pandas-nightly-numpy-nightly
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.13
test-conda-python-3.9
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5
test-conda-python-emscripten
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-debian-12-python-3-amd64
test-debian-12-python-3-i386
test-fedora-39-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

[raulcd]

The cython2 failure is related:

 In file included from /build/python/pyarrow/src/arrow/python/async.h:22,
from /build/python/build/temp.linux-x86_64-cpython-310/_parquet.cpp:830:
/build/python/pyarrow/src/arrow/python/common.h: In instantiation of 'arrow::py::SmartPtrNoGIL<SmartPtr, Ts>& arrow::py::SmartPtrNoGIL<SmartPtr, Ts>::operator=(V&&) [with V = std::unique_ptr<arrow::RecordBatchReader>&; SmartPtr = std::unique_ptr; Ts = {arrow::RecordBatchReader}]':
/build/python/build/temp.linux-x86_64-cpython-310/_parquet.cpp:25556:56: required from here
/build/python/pyarrow/src/arrow/python/common.h:263:20: error: use of deleted function 'std::unique_ptr<_Tp, _Dp>& std::unique_ptr<_Tp, _Dp>::operator=(const std::unique_ptr<_Tp, _Dp>&) [with _Tp = arrow::RecordBatchReader; _Dp = std::default_delete<arrow::RecordBatchReader>]'
263 | Base::operator=(std::forward<V>(v));
| ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
In file included from /opt/conda/envs/arrow/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/memory:78,
from /build/python/build/temp.linux-x86_64-cpython-310/_parquet.cpp:770:
/opt/conda/envs/arrow/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:523:19: note: declared here
523 | unique_ptr& operator=(const unique_ptr&) = delete;
| ^~~~~~~~

the following patch would work but I don't think this should be the fix, as I am pretty sure std::forward should be used here for other operators:

diff --git a/python/pyarrow/src/arrow/python/common.h b/python/pyarrow/src/arrow/python/common.h
index 4a7886695e..30acf0d6ed 100644
--- a/python/pyarrow/src/arrow/python/common.h+++ b/python/pyarrow/src/arrow/python/common.h@@ -260,7 +260,7 @@ class SmartPtrNoGIL : public SmartPtr<Ts...> {
template <typename V>
SmartPtrNoGIL& operator=(V&& v) {
auto release_guard = optional_gil_release();
- Base::operator=(std::forward<V>(v));+ Base::operator=(std::move(v));
return *this;
}

[raulcd]

@github-actions crossbow submit test-conda-python-3.10-cython2

[github-actions]

Revision: 07f7acdb3c41b8299ebe2903c39ab6c3c562faed

Submitted crossbow builds: ursacomputing/crossbow @ actions-3db166d478

Task	Status
test-conda-python-3.10-cython2

[raulcd]

@github-actions crossbow submit -g python

[raulcd]

@github-actions crossbow submit -g python

[github-actions]

Revision: babbe9fa049f5fcc1144f45150cae737aa8425bf

Submitted crossbow builds: ursacomputing/crossbow @ actions-f876b9ef9d

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-cython2
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest-numpy-latest
test-conda-python-3.10-substrait
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-1.26
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-pandas-nightly-numpy-nightly
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.13
test-conda-python-3.9
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5
test-conda-python-emscripten
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-debian-12-python-3-amd64
test-debian-12-python-3-i386
test-fedora-39-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

[raulcd]

@github-actions crossbow submit test-conda-python-3.10-cython2

[github-actions]

Revision: f292cc101be901b98b0d9357db5b564efee76de7

Submitted crossbow builds: ursacomputing/crossbow @ actions-58dca82b3c

Task	Status
test-conda-python-3.10-cython2

[pitrou]

Ok, so the shared_ptr version works but not the unique_ptr one? In this case, we can keep using the shared_ptr approach.

[pitrou]

By the way, why do we want to support Cython 2? @jorisvandenbossche

[jorisvandenbossche]

I would say we can drop the cython 2 support.

[raulcd]

I would say we can drop the cython 2 support.

I created an issue to track that as it is not the first time we say the same. I'll do and then I'll rebase this one. I'll ping to merge once CI is green again.

[raulcd]

@github-actions crossbow submit -g python

[github-actions]

Revision: 0a1d214

Submitted crossbow builds: ursacomputing/crossbow @ actions-74244302df

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest-numpy-latest
test-conda-python-3.10-substrait
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-1.26
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-pandas-nightly-numpy-nightly
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.13
test-conda-python-3.9
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5
test-conda-python-emscripten
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-debian-12-python-3-amd64
test-debian-12-python-3-i386
test-fedora-39-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

[raulcd]

CI failures are unrelated, they are failing on main and I've opened individual issues to track them.

@pitrou I've rebased after the required cython bump. This should be ready to review/merge now that there's no CI failure.

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 984519d.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 4 possible false positives for unstable benchmarks that are known to sometimes produce them.