[Python] Nightly wheels missing for Windows

[larsoner]

Describe the bug, including details regarding any error messages, version, and platform.

See

https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/pyarrow/

This suggests they were either removed on purpose (sorry if I missed the discussion!) or that the upload hasn't worked for about a month. Or at least I think ~1mo is how long the artifacts are retained there...

Component(s)

Python

[raulcd]

Thanks for raising the issue. There's a maximum number of artifacts retained that removes old wheels if they fail for ~4-5 days. The Windows wheels nightly jobs have been failing for 6 days. It requires some investigation, I won't be available to investigate for the rest of the week but the failures are due to some failures on 3 cython tests. If someone can investigate:
Failures example:
https://github.com/ursacomputing/crossbow/actions/runs/16745071346/job/47401564422

[AlenkaF]

I plan to do some investigation tomorrow (alongside the free-threaded build failures) and will share what I find.

[AlenkaF]

A new release of Visual Studio 2022 might be causing issues. See:

• Version 17.14.11, released on August 4th, 2025

• A note regarding the Windows SDK 10.0.20348 we use, which has been removed from the VS 2022 installer in the above version:

  The following Windows SDK versions have been removed from the Visual Studio 2022 installer: 10.0.18362.0, 10.0.20348.0 and 10.0.22000.0. If you previously installed one of these versions of the SDK using Visual Studio it will be uninstalled when you update. If your project targets any of these SDKs you may encounter a build error such as: The Windows SDK version 10.0.22000.0 was not found. Install the required version of Windows SDK or change the SDK version in the project property pages or by right-clicking the solution and selecting "Retarget solution". To resolve this, we recommend retargeting your project to 10.0.26100.0, or an earlier supported version if necessary. For a complete list of supported SDK versions please visit: https://developer.microsoft.com/windows/downloads/sdk-archive/. If you need to install an unsupported version of the SDK, you can find it here: https://developer.microsoft.com/windows/downloads/sdk-archive/index-legacy.

I plan to try updating the Python Windows wheel Dockerfiles in this PR to see if that helps.

[AlenkaF]

I have built PyArrow on a Windows machine and cannot reproduce the failure. I should have used the same MSVC, flags and Python version. The main difference I see is in the include paths.

Locally, the ucrt, um, shared, and other Windows Kits directories are present. But in the CI, these are missing — only ...\include folders appear.

Local build output

(pyarrow-dev) C:\Users\Public\Repositories\arrow\python>python -m pytest pyarrow/tests/test_cython.py -k test_cython_api -s
Package "tzdata" not found. Not setting TZDIR environment variable.
============================================ test session starts ============================================
platform win32 -- Python 3.12.11, pytest-8.4.1, pluggy-1.6.0
rootdir: C:\Users\Public\Repositories\arrow\python
configfile: setup.cfg
plugins: hypothesis-6.138.0
collected 2 items / 1 deselected / 1 selected

pyarrow\tests\test_cython.py Compiling pyarrow_cython_example.pyx because it changed.
[1/1] Cythonizing pyarrow_cython_example.pyx
Extension module: <setuptools.extension.Extension('pyarrow_cython_example') at 0x2a89e690d40> ['C:\\Users\\Inti\\.conda\\envs\\pyarrow-dev\\Lib\\site-packages\\numpy\\_core\\include', 'C:\\Users\\Public\\Repositories\\arrow\\python\\pyarrow\\include'] ['arrow_python', 'arrow'] ['C:\\Users\\Public\\Repositories\\arrow\\python\\pyarrow', 'C:/Users/Inti/.conda/envs/pyarrow-dev/Library/lib', 'C:\\Users\\Inti\\.conda\\envs\\pyarrow-dev\\Library\\lib']
running build_ext
building 'pyarrow_cython_example' extension
creating build\temp.win-amd64-cpython-312\Release
"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\Inti\.conda\envs\pyarrow-dev\Lib\site-packages\numpy\_core\include -IC:\Users\Public\Repositories\arrow\python\pyarrow\include -IC:\Users\Inti\.conda\envs\pyarrow-dev\include -IC:\Users\Inti\.conda\envs\pyarrow-dev\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.26100.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\cppwinrt" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.26100.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\cppwinrt" /EHsc /Tppyarrow_cython_example.cpp /Fobuild\temp.win-amd64-cpython-312\Release\pyarrow_cython_example.obj -D_ENABLE_EXTENDED_ALIGNED_STORAGE /std:c++17
pyarrow_cython_example.cpp
pyarrow_cython_example.cpp(13990): warning C4551: function call missing argument list
pyarrow_cython_example.cpp(13997): warning C4551: function call missing argument list
creating C:\Users\Inti\AppData\Local\Temp\pytest-of-Inti\pytest-1\test_cython_api0\build\lib.win-amd64-cpython-312
"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\Public\Repositories\arrow\python\pyarrow /LIBPATH:C:/Users/Inti/.conda/envs/pyarrow-dev/Library/lib /LIBPATH:C:\Users\Inti\.conda\envs\pyarrow-dev\Library\lib /LIBPATH:C:\Users\Inti\.conda\envs\pyarrow-dev\libs /LIBPATH:C:\Users\Inti\.conda\envs\pyarrow-dev /LIBPATH:C:\Users\Inti\.conda\envs\pyarrow-dev\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.26100.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.26100.0\\um\x64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.26100.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.26100.0\\um\x64" arrow_python.lib arrow.lib /EXPORT:PyInit_pyarrow_cython_example build\temp.win-amd64-cpython-312\Release\pyarrow_cython_example.obj /OUT:build\lib.win-amd64-cpython-312\pyarrow_cython_example.cp312-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-312\Release\pyarrow_cython_example.cp312-win_amd64.lib
 Creating library build\temp.win-amd64-cpython-312\Release\pyarrow_cython_example.cp312-win_amd64.lib and object build\temp.win-amd64-cpython-312\Release\pyarrow_cython_example.cp312-win_amd64.exp
Generating code
Finished generating code
copying build\lib.win-amd64-cpython-312\pyarrow_cython_example.cp312-win_amd64.pyd ->
.

====================================== 1 passed, 1 deselected in 9.80s ======================================

CI build output

"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Python312\Lib\site-packages\numpy\core\include -IC:\Python312\Lib\site-packages\pyarrow\include -IC:\Python312\include -IC:\Python312\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" /EHsc /Tppyarrow_cython_example.cpp /Fobuild\temp.win-amd64-cpython-312\Release\pyarrow_cython_example.obj -D_ENABLE_EXTENDED_ALIGNED_STORAGE /std:c++17

pyarrow_cython_example.cpp

C:\Python312\include\pyconfig.h(59): fatal error C1083: Cannot open include file: 'io.h': No such file or directory

This would suggests a problem with the Visual Studio environment on the CI— likely the Windows SDK is missing or not configured properly.

See:

• discuss.python.org: Where is io.h file used as an include in pyconfig.h (Python 3.12)
• Microsoft Q&A: io.h not found

Interestingly, C++ and PyArrow build successfully, but the Cython test fails, which is unexpected. Currently the only idea I have is to skip tests in case the header files are not found/Windows SDK is not installed.

[AlenkaF]

Thought at first that delvewheel versions were the same beteween builds but now I see there has been a bump:

• failing CI build https://github.com/ursacomputing/crossbow/actions/runs/17018910791/job/48245182645#step:7:7904
  delvewheel-1.11.1 pefile-2024.8.26
• successful CI build https://github.com/ursacomputing/crossbow/actions/runs/16718301087/job/47316555137#step:7:7905
  installed delvewheel-1.11.0 pefile-2024.8.26

the timing matches (5th of August) but the changelog shows nothing that would be connected:

https://github.com/adang1345/delvewheel/blob/master/CHANGELOG.md#1111-5-august-2025

• Miscellaneous README improvements.
• Fix UnicodeDecodeError when patching .py files containing Unicode characters.
• Don't strip off ending newline when inserting .py patch after docstring.

[AlenkaF]

The issue seems to be tackled here: #47357

[raulcd]

A PR that fixes the Windows wheels have been merged here:
#47367

Nightlies should be available on the next 24 hours (once the nightly job triggers and finishes). We can close this issue then.

[raulcd]

Windows wheels have been uploaded successfully:

https://anaconda.org/scientific-python-nightly-wheels/pyarrow/files