GH-48894: [Python][C++] Use base Azure::Core::RequestFailedException instead of final Azure::Storage::StorageException and set minimum nodejs on conda env to 16 for Azurite to work by raulcd · Pull Request #48895 · apache/arrow

raulcd · 2026-01-19T11:37:23Z

Rationale for this change

nodejs 12 is currently being installed on conda. CI jobs are failing and or segfaulting due to azurite failing with old versions.

2026-01-13T18:32:39.6961900Z #15 [ 9/11] RUN /arrow/ci/scripts/install_azurite.sh
2026-01-13T18:32:39.9624124Z #15 0.417 Node.js version = v12.4.0
2026-01-13T18:32:42.2087322Z #15 2.663 npm WARN deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported
2026-01-13T18:32:42.3917601Z #15 2.846 npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
2026-01-13T18:32:51.4870197Z #15 11.94 npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
2026-01-13T18:32:51.7035681Z #15 12.01 npm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
2026-01-13T18:33:02.1406491Z #15 22.59 /opt/conda/envs/arrow/bin/azurite -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/azurite.js
2026-01-13T18:33:02.3841290Z #15 22.60 /opt/conda/envs/arrow/bin/azurite-queue -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/queue/main.js
2026-01-13T18:33:02.3842792Z #15 22.60 /opt/conda/envs/arrow/bin/azurite-blob -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/blob/main.js
2026-01-13T18:33:02.3844216Z #15 22.60 /opt/conda/envs/arrow/bin/azurite-table -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/table/main.js
2026-01-13T18:33:02.3846002Z #15 22.66 npm WARN applicationinsights@2.9.8 requires a peer of applicationinsights-native-metrics@* but none is installed. You must install peer dependencies yourself.
2026-01-13T18:33:02.3847278Z #15 22.66 2026-01-13T18:33:02.3847564Z #15 22.66 + azurite@3.35.0
2026-01-13T18:33:02.3848038Z #15 22.66 added 376 packages from 296 contributors in 20.644s
2026-01-13T18:33:02.3848830Z #15 22.69 /opt/conda/envs/arrow/bin/azurite
2026-01-13T18:33:02.8929329Z #15 23.35 /opt/conda/envs/arrow/lib/node_modules/azurite/node_modules/fs-extra/lib/util/async.js:14
2026-01-13T18:33:02.8930231Z #15 23.35 (err) => err ?? new Error('unknown error')
2026-01-13T18:33:02.8930740Z #15 23.35 ^

The job on PyArrow was segfaulting due to an Exception being thrown but not catch. In general we were using Azure::Storage::StorageException but Azure::Core::Http::TransportException could also be thrown on some cases.
Bot are final but inherit from Azure::Core::RequestFailedException.

What changes are included in this PR?

Pin minimum nodejs version to 16 so the failure doesn't happen again.
Update catching Azure::Storage::StorageException to Azure::Core::RequestFailedException so Azure::Core::Http::TransportException is also catch.

Are these changes tested?

Yes on CI.

Are there any user-facing changes?

No

GitHub Issue: [Python] Conda jobs segfault on Azure tests due to old nodejs version being installed #48894

…urite to work

raulcd · 2026-01-19T11:37:53Z

@github-actions crossbow submit test-conda-*

github-actions · 2026-01-19T11:38:12Z

⚠️ GitHub issue #48894has been automatically assigned in GitHub to PR creator.

github-actions · 2026-01-19T11:40:46Z

Revision: 3fabe1d

Submitted crossbow builds: ursacomputing/crossbow @ actions-cd25c6296c

Task	Status
test-conda-cpp
test-conda-cpp-valgrind
test-conda-python-3.10
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-1.3.4-numpy-1.21.2
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.12-pandas-latest-numpy-1.26
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-nightly-numpy-nightly
test-conda-python-3.13-pandas-upstream_devel-numpy-nightly
test-conda-python-3.14
test-conda-python-emscripten

raulcd · 2026-01-19T13:21:13Z

Moving this back to draft to investigate the segfault before fixing the nodejs version.

…age::StorageException so we also catch Azure::Core::Http::TransportException

raulcd · 2026-01-19T17:20:03Z

@github-actions crossbow submit -g cpp

github-actions · 2026-01-19T17:22:33Z

Revision: 3ccfdb7

Submitted crossbow builds: ursacomputing/crossbow @ actions-d7c2a57735

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-12-cpp-amd64
test-debian-12-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

raulcd · 2026-01-20T08:43:22Z

@pitrou is this what you expected? About factoring out the exception-catching code we can follow on a different issue if you want, I am unsure what that would require.
I initially started this PR to fix our current CI failures only.

pitrou · 2026-01-20T09:11:26Z

@pitrou is this what you expected?

Yes, it is. Did you check that it averted the crashes by turning them into proper errors?

About factoring out the exception-catching code we can follow on a different issue if you want, I am unsure what that would require.

Looking at the code, I'm not sure there's much to do about it, actually.

raulcd · 2026-01-20T09:20:35Z

@github-actions crossbow submit test-conda-python-3.13

github-actions · 2026-01-20T09:22:46Z

Revision: b9b1ab6

Submitted crossbow builds: ursacomputing/crossbow @ actions-35c166620c

Task	Status
test-conda-python-3.13

This reverts commit b9b1ab6.

raulcd · 2026-01-20T09:32:27Z

@github-actions crossbow submit test-conda-python-3.13

github-actions · 2026-01-20T09:34:41Z

Revision: a86bb98

Submitted crossbow builds: ursacomputing/crossbow @ actions-6ceafde0b6

Task	Status
test-conda-python-3.13

raulcd · 2026-01-20T09:40:25Z

Did you check that it averted the crashes by turning them into proper errors?

Yes, I did locally but I've pushed a job without the nodejs bump so we can see it there. Pytest finishes running all tests, it doesn't crash anymore but tests fail as expected:
https://github.com/ursacomputing/crossbow/actions/runs/21166071337/job/60871016940#step:6:6596

opt/conda/envs/arrow/lib/python3.13/site-packages/pyarrow/tests/test_fs.py:329: inazurefsfs.create_dir(container)
pyarrow/_fs.pyx:638: inpyarrow._fs.FileSystem.create_dir
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > ???
EOSError: Failedtocreateacontainer: pyarrow-filesystem: http://127.0.0.1:35357/devstoreaccount1/pyarrow-filesystemAzureError: [] Failtogetanewconnectionfor: http://127.0.0.1:35357.Couldnotconnecttoserver

conbench-apache-arrow · 2026-01-20T18:37:27Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c183167.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 14 possible false positives for unstable benchmarks that are known to sometimes produce them.

…instead of final Azure::Storage::StorageException and set minimum nodejs on conda env to 16 for Azurite to work (#48895) ### Rationale for this change nodejs 12 is currently being installed on conda. CI jobs are failing and or segfaulting due to azurite failing with old versions. ``` 2026-01-13T18:32:39.6961900Z #15 [ 9/11] RUN /arrow/ci/scripts/install_azurite.sh 2026-01-13T18:32:39.9624124Z #15 0.417 Node.js version = v12.4.0 2026-01-13T18:32:42.2087322Z #15 2.663 npm WARN deprecated rimraf@ 3.0.2: Rimraf versions prior to v4 are no longer supported 2026-01-13T18:32:42.3917601Z #15 2.846 npm WARN deprecated uuid@ 3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. 2026-01-13T18:32:51.4870197Z #15 11.94 npm WARN deprecated glob@ 7.2.3: Glob versions prior to v9 are no longer supported 2026-01-13T18:32:51.7035681Z #15 12.01 npm WARN deprecated inflight@ 1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful. 2026-01-13T18:33:02.1406491Z #15 22.59 /opt/conda/envs/arrow/bin/azurite -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/azurite.js 2026-01-13T18:33:02.3841290Z #15 22.60 /opt/conda/envs/arrow/bin/azurite-queue -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/queue/main.js 2026-01-13T18:33:02.3842792Z #15 22.60 /opt/conda/envs/arrow/bin/azurite-blob -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/blob/main.js 2026-01-13T18:33:02.3844216Z #15 22.60 /opt/conda/envs/arrow/bin/azurite-table -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/table/main.js 2026-01-13T18:33:02.3846002Z #15 22.66 npm WARN applicationinsights@ 2.9.8 requires a peer of applicationinsights-native-metrics@* but none is installed. You must install peer dependencies yourself. 2026-01-13T18:33:02.3847278Z #15 22.66 2026-01-13T18:33:02.3847564Z #15 22.66 + azurite@ 3.35.0 2026-01-13T18:33:02.3848038Z #15 22.66 added 376 packages from 296 contributors in 20.644s 2026-01-13T18:33:02.3848830Z #15 22.69 /opt/conda/envs/arrow/bin/azurite 2026-01-13T18:33:02.8929329Z #15 23.35 /opt/conda/envs/arrow/lib/node_modules/azurite/node_modules/fs-extra/lib/util/async.js:14 2026-01-13T18:33:02.8930231Z #15 23.35 (err) => err ?? new Error('unknown error') 2026-01-13T18:33:02.8930740Z #15 23.35 ^ ``` The job on PyArrow was segfaulting due to an Exception being thrown but not catch. In general we were using `Azure::Storage::StorageException` but `Azure::Core::Http::TransportException` could also be thrown on some cases. Bot are final but inherit from `Azure::Core::RequestFailedException`. ### What changes are included in this PR? - Pin minimum nodejs version to 16 so the failure doesn't happen again. - Update catching `Azure::Storage::StorageException` to `Azure::Core::RequestFailedException` so `Azure::Core::Http::TransportException` is also catch. ### Are these changes tested? Yes on CI. ### Are there any user-facing changes? No * GitHub Issue: #48894 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>

## Description The main "fix" here is to bump arrow to pull in apache/arrow#48895 , which could lead to escaped Azure transport exceptions from arrow filesystem operations. In order to be more robust against unexpected exceptions, we also copy their trick and slightly broaden the caught exception to `Azure::Core::RequestFailedException`, and cover all function calls that go into the Azure SDK. https://claude.ai/code/session_01MfTGpRoNmmvrHGYLRwWVTa --------- Co-authored-by: tobim <tobim@fastmail.fm>

…ption instead of final Azure::Storage::StorageException and set minimum nodejs on conda env to 16 for Azurite to work (apache#48895) ### Rationale for this change nodejs 12 is currently being installed on conda. CI jobs are failing and or segfaulting due to azurite failing with old versions. ``` 2026-01-13T18:32:39.6961900Z apache#15 [ 9/11] RUN /arrow/ci/scripts/install_azurite.sh 2026-01-13T18:32:39.9624124Z apache#15 0.417 Node.js version = v12.4.0 2026-01-13T18:32:42.2087322Z apache#15 2.663 npm WARN deprecated rimraf@ 3.0.2: Rimraf versions prior to v4 are no longer supported 2026-01-13T18:32:42.3917601Z apache#15 2.846 npm WARN deprecated uuid@ 3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. 2026-01-13T18:32:51.4870197Z apache#15 11.94 npm WARN deprecated glob@ 7.2.3: Glob versions prior to v9 are no longer supported 2026-01-13T18:32:51.7035681Z apache#15 12.01 npm WARN deprecated inflight@ 1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful. 2026-01-13T18:33:02.1406491Z apache#15 22.59 /opt/conda/envs/arrow/bin/azurite -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/azurite.js 2026-01-13T18:33:02.3841290Z apache#15 22.60 /opt/conda/envs/arrow/bin/azurite-queue -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/queue/main.js 2026-01-13T18:33:02.3842792Z apache#15 22.60 /opt/conda/envs/arrow/bin/azurite-blob -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/blob/main.js 2026-01-13T18:33:02.3844216Z apache#15 22.60 /opt/conda/envs/arrow/bin/azurite-table -> /opt/conda/envs/arrow/lib/node_modules/azurite/dist/src/table/main.js 2026-01-13T18:33:02.3846002Z apache#15 22.66 npm WARN applicationinsights@ 2.9.8 requires a peer of applicationinsights-native-metrics@* but none is installed. You must install peer dependencies yourself. 2026-01-13T18:33:02.3847278Z apache#15 22.66 2026-01-13T18:33:02.3847564Z apache#15 22.66 + azurite@ 3.35.0 2026-01-13T18:33:02.3848038Z apache#15 22.66 added 376 packages from 296 contributors in 20.644s 2026-01-13T18:33:02.3848830Z apache#15 22.69 /opt/conda/envs/arrow/bin/azurite 2026-01-13T18:33:02.8929329Z apache#15 23.35 /opt/conda/envs/arrow/lib/node_modules/azurite/node_modules/fs-extra/lib/util/async.js:14 2026-01-13T18:33:02.8930231Z apache#15 23.35 (err) => err ?? new Error('unknown error') 2026-01-13T18:33:02.8930740Z apache#15 23.35 ^ ``` The job on PyArrow was segfaulting due to an Exception being thrown but not catch. In general we were using `Azure::Storage::StorageException` but `Azure::Core::Http::TransportException` could also be thrown on some cases. Bot are final but inherit from `Azure::Core::RequestFailedException`. ### What changes are included in this PR? - Pin minimum nodejs version to 16 so the failure doesn't happen again. - Update catching `Azure::Storage::StorageException` to `Azure::Core::RequestFailedException` so `Azure::Core::Http::TransportException` is also catch. ### Are these changes tested? Yes on CI. ### Are there any user-facing changes? No * GitHub Issue: apache#48894 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>

apacheGH-48894: [Python] Set minimum nodejs on conda env to 16 for Az…
3fabe1d
…urite to work

github-actionsBot added the awaiting committer review Awaiting committer review label Jan 19, 2026

raulcd marked this pull request as ready for review January 19, 2026 12:11

raulcd requested review from assignUser, jonkeane and kou as code owners January 19, 2026 12:11

raulcd mentioned this pull request Jan 19, 2026
Fix CI conda-forge/arrow-cpp-feedstock#1913
Merged

raulcd marked this pull request as draft January 19, 2026 13:20

Catch base Azure::Core::RequestFailedException instead of Azure::Stor…
3ccfdb7
…age::StorageException so we also catch Azure::Core::Http::TransportException

github-actionsBot added the Component: C++ label Jan 19, 2026

raulcd changed the title ~~GH-48894: [Python] Set minimum nodejs on conda env to 16 for Azurite to work~~GH-48894: [Python][C++] Set minimum nodejs on conda env to 16 for Azurite to work and use base Azure::Core::RequestFailedException instead of final Azure::Storage::StorageExceptionJan 20, 2026

raulcd marked this pull request as ready for review January 20, 2026 08:40

raulcd requested a review from pitrou January 20, 2026 08:43

Remove pin for nodejs to see new Error (not crash) on CI
b9b1ab6

Revert "Remove pin for nodejs to see new Error (not crash) on CI"
a86bb98
This reverts commit b9b1ab6.

pitrou approved these changes Jan 20, 2026
View reviewed changes

pitrou mentioned this pull request Jan 20, 2026
GH-48858: [C++][Parquet] Avoid re-serializing footer for signature verification #48859
Merged

raulcd merged commit c183167 into apache:mainJan 20, 2026
53 checks passed

raulcd removed the awaiting committer review Awaiting committer review label Jan 20, 2026

raulcd mentioned this pull request Jan 20, 2026
[Python] Conda jobs segfault on Azure tests due to old nodejs version being installed #48894
Closed

raulcd deleted the GH-48894 branch January 20, 2026 14:32

pitrou added the backport-candidate label Jan 29, 2026

raulcd removed the backport-candidate label Feb 3, 2026

lava mentioned this pull request Apr 8, 2026
Handle exceptions in Azure Blob Storage read/write operations tenzir/tenzir#5993
Merged

Uh oh!

Conversation

raulcd commented Jan 19, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

raulcd commented Jan 19, 2026

Uh oh!

github-actionsBot commented Jan 19, 2026

Uh oh!

github-actionsBot commented Jan 19, 2026

Uh oh!

raulcd commented Jan 19, 2026

Uh oh!

raulcd commented Jan 19, 2026

Uh oh!

github-actionsBot commented Jan 19, 2026

Uh oh!

raulcd commented Jan 20, 2026

Uh oh!

pitrou commented Jan 20, 2026

Uh oh!

raulcd commented Jan 20, 2026

Uh oh!

github-actionsBot commented Jan 20, 2026

Uh oh!

raulcd commented Jan 20, 2026

Uh oh!

github-actionsBot commented Jan 20, 2026

Uh oh!

raulcd commented Jan 20, 2026

Uh oh!

Uh oh!

conbench-apache-arrowBot commented Jan 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

raulcd commented Jan 19, 2026•
edited
Loading