GH-51138: [Python] Fix ccache efficiency by pitrou · Pull Request #51139 · apache/arrow

pitrou · 2026-09-02T13:34:05Z

Rationale for this change

Our current ccache support for PyArrow currently has two issues:

The custom configuration is set using environment variables, but those are not passed to the build processes according to the CMake doc (I have verified this by enabling ccache logging to see the configuration values for each build command)
Cython-generated C++ sources are written in the temporary build directory (because of build isolation), and therefore their path changes everytime. Consequently, ccache would fail reusing previous compilation results.

What changes are included in this PR?

Pass custom ccache configuration using command-line arguments, not environment variables.
Set the ccache configuration variable base_dir to the temporary build directory so that ccache strips away the build directory and hits previously cached results obtained from a different build directory.

Are these changes tested?

Yes, locally I confirmed that ccache now efficiently reuses compilation outputs for Cython-generated C++ sources.

Before:

real 0m50,313s
user 4m22,991s
sys 0m24,805s

After:

real 0m18,698s
user 1m14,727s
sys 0m7,518s

Are there any user-facing changes?

No.

GitHub Issue: [Python] ccache not efficient #51138

github-actions · 2026-09-02T13:35:47Z

⚠️ GitHub issue #51138has been automatically assigned in GitHub to PR creator.

pitrou · 2026-09-02T14:51:36Z

@github-actions crossbow submit -g python

github-actions · 2026-09-02T14:54:49Z

Revision: a66cc75

Submitted crossbow builds: ursacomputing/crossbow @ actions-290df36268

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hdfs-2.9.2
test-conda-python-3.11-hdfs-3.2.1
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-2.2.2-numpy-2.0.2
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-latest-numpy-2.1.3
test-conda-python-3.13-pandas-latest-numpy-latest
test-conda-python-3.14
test-conda-python-3.14-cpython-debug
test-conda-python-3.14-pandas-nightly-numpy-nightly
test-conda-python-3.14-pandas-upstream_devel-numpy-nightly
test-conda-python-emscripten
test-debian-13-python-3-amd64
test-debian-13-python-3-i386
test-fedora-42-python-3
test-ubuntu-24.04-python-3

rok · 2026-09-02T15:22:48Z

@github-actions crossbow submit test-ubuntu-22.04-cpp -p image=ubuntu-python

github-actions · 2026-09-02T15:25:19Z

Revision: a66cc75

Submitted crossbow builds: ursacomputing/crossbow @ actions-ba899c1522

Task	Status
test-ubuntu-22.04-cpp

pitrou · 2026-09-02T16:22:28Z

@github-actions crossbow submit test-ubuntu-22.04-cpp -p image=ubuntu-python

What's this @rok ?

rok · 2026-09-02T17:30:57Z

+ # 2. Set ccache base_dir to the build output directory as it is typically
+ # a temporary directory, and would otherwise fail caching because of
+ # using different paths everytime.
+ set(ccache_command ${CCACHE_FOUND} keep_comments_cpp=true base_dir=${CMAKE_BINARY_DIR})


As per my 🤖 ccache pre-4.8 does not accept compiler options (keep_comments_cpp=true in this case). And testing on an older ccache I get:
> ccache keep_comments_cpp=true base_dir=/tmp gcc --versionccache: error: Could not find compiler "keep_comments_cpp=true" in PATH
So users with old ccache building pyarrow would have this crash.

Hmm, are there any Python builds which use an old ccache?

I think no, but users might. I think this is obscure enough to ignore.

If there was an easy way to query the ccache version we could work around this to ensure a better UX. But I'm not sure how to do that. @kou ?

(for the record, ccache 4.8 was released in March 2023)

@kou How about cmake -E env as suggested by @rok above?
$ cmake -E env CCACHE_BASEDIR=/top -- ccache -p |\grep base_dir(environment) base_dir = /top

It can avoid the error but meaningless ccache command is still used (ccache storage is consumed). If it's acceptable, we can use the cmake -E env approach.

Why "meaningless"?

Because:
Cython-generated C++ sources are written in the temporary build directory (because of build isolation), and therefore their path changes everytime. Consequently, ccache would fail reusing previous compilation results.

We're setting CCACHE_BASEDIR precisely to avoid that.

pitrou · 2026-09-02T17:32:30Z

@github-actions crossbow submit wheelcp311*

github-actions · 2026-09-02T17:35:17Z

Revision: a66cc75

Submitted crossbow builds: ursacomputing/crossbow @ actions-131e0bf55a

Task	Status
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-manylinux-2-28-cp311-cp311-amd64
wheel-manylinux-2-28-cp311-cp311-arm64
wheel-musllinux-1-2-cp311-cp311-amd64
wheel-musllinux-1-2-cp311-cp311-arm64
wheel-windows-cp311-cp311-amd64

pitrou · 2026-09-03T08:33:57Z

@github-actions crossbow submit -g python

pitrou · 2026-09-03T08:34:04Z

@github-actions crossbow submit wheelcp311*

github-actions · 2026-09-03T08:37:27Z

Revision: a1df242

Submitted crossbow builds: ursacomputing/crossbow @ actions-db9b456f6a

Task	Status
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-manylinux-2-28-cp311-cp311-amd64
wheel-manylinux-2-28-cp311-cp311-arm64
wheel-musllinux-1-2-cp311-cp311-amd64
wheel-musllinux-1-2-cp311-cp311-arm64
wheel-windows-cp311-cp311-amd64

github-actions · 2026-09-03T08:38:09Z

Revision: a1df242

Submitted crossbow builds: ursacomputing/crossbow @ actions-695eab7643

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hdfs-2.9.2
test-conda-python-3.11-hdfs-3.2.1
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-2.2.2-numpy-2.0.2
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-latest-numpy-2.1.3
test-conda-python-3.13-pandas-latest-numpy-latest
test-conda-python-3.14
test-conda-python-3.14-cpython-debug
test-conda-python-3.14-pandas-nightly-numpy-nightly
test-conda-python-3.14-pandas-upstream_devel-numpy-nightly
test-conda-python-emscripten
test-debian-13-python-3-amd64
test-debian-13-python-3-i386
test-fedora-42-python-3
test-ubuntu-24.04-python-3

pitrou · 2026-09-03T09:15:48Z

I'm going to merge as CI failures are unrelated.

github-actionsBot added the awaiting review Awaiting review label Sep 2, 2026

github-actionsBot added the Component: Python label Sep 2, 2026

pitrou force-pushed the gh51138-python-ccache branch from 9a5a1b7 to a66cc75Compare September 2, 2026 13:46

pitrou mentioned this pull request Sep 2, 2026
[Python] Adopt new features and bugfixes from Cython 3.1 #47467
Open

pitrou marked this pull request as ready for review September 2, 2026 14:51

pitrou requested review from AlenkaF, assignUser, jonkeane, kou, raulcd and rok as code owners September 2, 2026 14:51

rok reviewed Sep 2, 2026
View reviewed changes

github-actionsBot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Sep 2, 2026

rok approved these changes Sep 2, 2026
View reviewed changes

github-actionsBot added awaiting merge Awaiting merge awaiting changes Awaiting changes and removed awaiting changes Awaiting changes awaiting merge Awaiting merge labels Sep 2, 2026

kou added the CI: Extra: CUDA Run extra CUDA CI label Sep 3, 2026

pitrou added 2 commits September 3, 2026 10:28

apacheGH-51138: [Python] Fix ccache efficiency
298337a

Use cmake -E env to set ccache configuration options
a1df242

pitrou force-pushed the gh51138-python-ccache branch from a66cc75 to a1df242Compare September 3, 2026 08:33

github-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Sep 3, 2026

github-actionsBot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Sep 3, 2026

pitrou merged commit 503cee9 into apache:mainSep 3, 2026
42 of 44 checks passed

pitrou removed the awaiting changes Awaiting changes label Sep 3, 2026

pitrou deleted the gh51138-python-ccache branch September 3, 2026 09:16

pitrou mentioned this pull request Sep 3, 2026
[Python] ccache not efficient #51138
Closed

Uh oh!

Conversation

pitrou commented Sep 2, 2026• edited by github-actions BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actionsBot commented Sep 2, 2026

Uh oh!

pitrou commented Sep 2, 2026

Uh oh!

github-actionsBot commented Sep 2, 2026

Uh oh!

rok commented Sep 2, 2026

Uh oh!

github-actionsBot commented Sep 2, 2026

Uh oh!

pitrou commented Sep 2, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pitrou commented Sep 2, 2026

Uh oh!

github-actionsBot commented Sep 2, 2026

Uh oh!

pitrou commented Sep 3, 2026

Uh oh!

pitrou commented Sep 3, 2026

Uh oh!

github-actionsBot commented Sep 3, 2026

Uh oh!

github-actionsBot commented Sep 3, 2026

Uh oh!

pitrou commented Sep 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pitrou commented Sep 2, 2026•
edited by github-actions Bot
Loading