GH-35193: [Python][Packaging] Enable GCS on Windows wheels#35255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
assignUser merged 8 commits into from
May 16, 2023
Merged

GH-35193: [Python][Packaging] Enable GCS on Windows wheels #35255

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3d1dd94
WIP: GH-35193: [Python][Packaging] Enable GCS on Windows wheel
raulcd Apr 20, 2023
b2b891a
Try installing GCS testbench on Windows to run tests
raulcd Apr 20, 2023
50efe4e
Pin urllib3
raulcd May 11, 2023
30bad90
Upgrade testbench to v0.36.0
raulcd May 11, 2023
7466029
Try to add PYTHON environment variable to point to valid python execu…
raulcd May 12, 2023
bbfb7db
Set executable name to python
raulcd May 12, 2023
e1d51a8
Better comment code
raulcd May 12, 2023
cf41c23
Change variable name to be consistent with similar ones
raulcd May 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions ci/appveyor-cpp-build.bat
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,9 @@ set CMAKE_ARGS=-DARROW_DEPENDENCY_SOURCE=CONDA -DARROW_WITH_BZ2=ON
@rem Enable warnings-as-errors
set ARROW_CXXFLAGS=/WX /MP

@rem Install GCS testbench
call %CD%\ci\scripts\install_gcs_testbench.bat

@rem
@rem Build and test Arrow C++ libraries (including Parquet)
@rem
Expand DownExpand Up@@ -70,6 +73,7 @@ cmake -G "%GENERATOR%" %CMAKE_ARGS% ^
-DARROW_FLIGHT=%ARROW_BUILD_FLIGHT% ^
-DARROW_FLIGHT_SQL=%ARROW_BUILD_FLIGHT_SQL% ^
-DARROW_GANDIVA=%ARROW_BUILD_GANDIVA% ^
-DARROW_GCS=%ARROW_GCS% ^
-DARROW_HDFS=ON ^
-DARROW_JSON=ON ^
-DARROW_MIMALLOC=ON ^
Expand DownExpand Up@@ -97,6 +101,9 @@ cmake --build . --target install --config Release || exit /B
@rem For ORC C++
set TZDIR=%CONDA_PREFIX%\share\zoneinfo

@rem For finding Python executable for GCS tests
set PYTHON=python

ctest --output-on-failure || exit /B

popd
Expand All@@ -114,6 +121,7 @@ set PYARROW_WITH_ACERO=ON
set PYARROW_WITH_DATASET=ON
set PYARROW_WITH_FLIGHT=%ARROW_BUILD_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_BUILD_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_PARQUET=ON
set PYARROW_WITH_PARQUET_ENCRYPTION=ON
set PYARROW_WITH_S3=%ARROW_S3%
Expand Down
25 changes: 25 additions & 0 deletions ci/scripts/install_gcs_testbench.bat
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
@rem Licensed to the Apache Software Foundation (ASF) under one
@rem or more contributor license agreements. See the NOTICE file
@rem distributed with this work for additional information
@rem regarding copyright ownership. The ASF licenses this file
@rem to you under the Apache License, Version 2.0 (the
@rem "License"); you may not use this file except in compliance
@rem with the License. You may obtain a copy of the License at
@rem
@rem http://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing,
@rem software distributed under the License is distributed on an
@rem "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@rem KIND, either express or implied. See the License for the
@rem specific language governing permissions and limitations
@rem under the License.

@echo on

set GCS_TESTBENCH_VERSION="v0.36.0"

@REM Install GCS testbench %GCS_TESTBENCH_VERSION%
python -m pip install ^
"https://github.com/googleapis/storage-testbench/archive/%GCS_TESTBENCH_VERSION%.tar.gz" ^
|| exit /B 1
3 changes: 3 additions & 0 deletions ci/scripts/python_wheel_windows_build.bat
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@ set ARROW_ACERO=ON
set ARROW_DATASET=ON
set ARROW_FLIGHT=ON
set ARROW_GANDIVA=OFF
set ARROW_GCS=ON
set ARROW_HDFS=ON
set ARROW_ORC=OFF
set ARROW_PARQUET=ON
Expand DownExpand Up@@ -73,6 +74,7 @@ cmake ^
-DARROW_FILESYSTEM=ON ^
-DARROW_FLIGHT=%ARROW_FLIGHT% ^
-DARROW_GANDIVA=%ARROW_GANDIVA% ^
-DARROW_GCS=%ARROW_GCS% ^
-DARROW_HDFS=%ARROW_HDFS% ^
-DARROW_JSON=ON ^
-DARROW_MIMALLOC=%ARROW_MIMALLOC% ^
Expand DownExpand Up@@ -111,6 +113,7 @@ set PYARROW_WITH_ACERO=%ARROW_ACERO%
set PYARROW_WITH_DATASET=%ARROW_DATASET%
set PYARROW_WITH_FLIGHT=%ARROW_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_HDFS=%ARROW_HDFS%
set PYARROW_WITH_ORC=%ARROW_ORC%
set PYARROW_WITH_PARQUET=%ARROW_PARQUET%
Expand Down
5 changes: 5 additions & 0 deletions ci/scripts/python_wheel_windows_test.bat
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -22,6 +22,7 @@ set PYARROW_TEST_CYTHON=ON
set PYARROW_TEST_DATASET=ON
set PYARROW_TEST_FLIGHT=ON
set PYARROW_TEST_GANDIVA=OFF
set PYARROW_TEST_GCS=ON
set PYARROW_TEST_HDFS=ON
set PYARROW_TEST_ORC=OFF
set PYARROW_TEST_PARQUET=ON
Expand All@@ -38,11 +39,15 @@ set PARQUET_TEST_DATA=C:\arrow\submodules\parquet-testing\data
@REM Install testing dependencies
pip install -r C:\arrow\python\requirements-wheel-test.txt || exit /B 1

@REM Install GCS testbench
call "C:\arrow\ci\scripts\install_gcs_testbench.bat"

@REM Install the built wheels
python -m pip install --no-index --find-links=C:\arrow\python\dist\ pyarrow || exit /B 1

@REM Test that the modules are importable
python -c "import pyarrow" || exit /B 1
python -c "import pyarrow._gcsfs" || exit /B 1
python -c "import pyarrow._hdfs" || exit /B 1
python -c "import pyarrow._s3fs" || exit /B 1
python -c "import pyarrow.csv" || exit /B 1
Expand Down
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
GH-35193: [Python][Packaging] Enable GCS on Windows wheels#35255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

assignUser
merged 8 commits into

from

May 16, 2023
 Merged


GH-35193: [Python][Packaging] Enable GCS on Windows wheels
#35255
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
3d1dd94

WIP: GH-35193: [Python][Packaging] Enable GCS on Windows wheel


raulcd Apr 20, 2023
b2b891a

Try installing GCS testbench on Windows to run tests


raulcd Apr 20, 2023
50efe4e

Pin urllib3


raulcd May 11, 2023
30bad90

Upgrade testbench to v0.36.0


raulcd May 11, 2023
7466029

Try to add PYTHON environment variable to point to valid python execu…


raulcd May 12, 2023
bbfb7db

Set executable name to python


raulcd May 12, 2023
e1d51a8

Better comment code


raulcd May 12, 2023
cf41c23

Change variable name to be consistent with similar ones


raulcd May 12, 2023

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


8 changes: 8 additions & 0 deletions
ci/appveyor-cpp-build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,9 @@ set CMAKE_ARGS=-DARROW_DEPENDENCY_SOURCE=CONDA -DARROW_WITH_BZ2=ON
@rem Enable warnings-as-errors
set ARROW_CXXFLAGS=/WX /MP

@rem Install GCS testbench
call %CD%\ci\scripts\install_gcs_testbench.bat

@rem
@rem Build and test Arrow C++ libraries (including Parquet)
@rem
Expand DownExpand Up@@ -70,6 +73,7 @@ cmake -G "%GENERATOR%" %CMAKE_ARGS% ^
 -DARROW_FLIGHT=%ARROW_BUILD_FLIGHT% ^
 -DARROW_FLIGHT_SQL=%ARROW_BUILD_FLIGHT_SQL% ^
 -DARROW_GANDIVA=%ARROW_BUILD_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=ON ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=ON ^
Expand DownExpand Up@@ -97,6 +101,9 @@ cmake --build . --target install --config Release || exit /B
@rem For ORC C++
set TZDIR=%CONDA_PREFIX%\share\zoneinfo

@rem For finding Python executable for GCS tests
set PYTHON=python

ctest --output-on-failure || exit /B

popd
Expand All@@ -114,6 +121,7 @@ set PYARROW_WITH_ACERO=ON
set PYARROW_WITH_DATASET=ON
set PYARROW_WITH_FLIGHT=%ARROW_BUILD_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_BUILD_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_PARQUET=ON
set PYARROW_WITH_PARQUET_ENCRYPTION=ON
set PYARROW_WITH_S3=%ARROW_S3%
Expand Down

25 changes: 25 additions & 0 deletions
ci/scripts/install_gcs_testbench.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
@rem Licensed to the Apache Software Foundation (ASF) under one
@rem or more contributor license agreements. See the NOTICE file
@rem distributed with this work for additional information
@rem regarding copyright ownership. The ASF licenses this file
@rem to you under the Apache License, Version 2.0 (the
@rem "License"); you may not use this file except in compliance
@rem with the License. You may obtain a copy of the License at
@rem
@rem http://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing,
@rem software distributed under the License is distributed on an
@rem "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@rem KIND, either express or implied. See the License for the
@rem specific language governing permissions and limitations
@rem under the License.

@echo on

set GCS_TESTBENCH_VERSION="v0.36.0"

@REM Install GCS testbench %GCS_TESTBENCH_VERSION%
python -m pip install ^
 "https://github.com/googleapis/storage-testbench/archive/%GCS_TESTBENCH_VERSION%.tar.gz" ^
 || exit /B 1

3 changes: 3 additions & 0 deletions
ci/scripts/python_wheel_windows_build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@ set ARROW_ACERO=ON
set ARROW_DATASET=ON
set ARROW_FLIGHT=ON
set ARROW_GANDIVA=OFF
set ARROW_GCS=ON
set ARROW_HDFS=ON
set ARROW_ORC=OFF
set ARROW_PARQUET=ON
Expand DownExpand Up@@ -73,6 +74,7 @@ cmake ^
 -DARROW_FILESYSTEM=ON ^
 -DARROW_FLIGHT=%ARROW_FLIGHT% ^
 -DARROW_GANDIVA=%ARROW_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=%ARROW_HDFS% ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=%ARROW_MIMALLOC% ^
Expand DownExpand Up@@ -111,6 +113,7 @@ set PYARROW_WITH_ACERO=%ARROW_ACERO%
set PYARROW_WITH_DATASET=%ARROW_DATASET%
set PYARROW_WITH_FLIGHT=%ARROW_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_HDFS=%ARROW_HDFS%
set PYARROW_WITH_ORC=%ARROW_ORC%
set PYARROW_WITH_PARQUET=%ARROW_PARQUET%
Expand Down

5 changes: 5 additions & 0 deletions
ci/scripts/python_wheel_windows_test.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -22,6 +22,7 @@ set PYARROW_TEST_CYTHON=ON
set PYARROW_TEST_DATASET=ON
set PYARROW_TEST_FLIGHT=ON
set PYARROW_TEST_GANDIVA=OFF
set PYARROW_TEST_GCS=ON
set PYARROW_TEST_HDFS=ON
set PYARROW_TEST_ORC=OFF
set PYARROW_TEST_PARQUET=ON
Expand All@@ -38,11 +39,15 @@ set PARQUET_TEST_DATA=C:\arrow\submodules\parquet-testing\data
@REM Install testing dependencies
pip install -r C:\arrow\python\requirements-wheel-test.txt || exit /B 1

@REM Install GCS testbench
call "C:\arrow\ci\scripts\install_gcs_testbench.bat"

@REM Install the built wheels
python -m pip install --no-index --find-links=C:\arrow\python\dist\ pyarrow || exit /B 1 

@REM Test that the modules are importable
python -c "import pyarrow" || exit /B 1
python -c "import pyarrow._gcsfs" || exit /B 1
python -c "import pyarrow._hdfs" || exit /B 1 
python -c "import pyarrow._s3fs" || exit /B 1
python -c "import pyarrow.csv" || exit /B 1
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-35193: [Python][Packaging] Enable GCS on Windows wheels#35255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

assignUser
merged 8 commits into

from

May 16, 2023
 Merged


GH-35193: [Python][Packaging] Enable GCS on Windows wheels
#35255
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
3d1dd94

WIP: GH-35193: [Python][Packaging] Enable GCS on Windows wheel


raulcd Apr 20, 2023
b2b891a

Try installing GCS testbench on Windows to run tests


raulcd Apr 20, 2023
50efe4e

Pin urllib3


raulcd May 11, 2023
30bad90

Upgrade testbench to v0.36.0


raulcd May 11, 2023
7466029

Try to add PYTHON environment variable to point to valid python execu…


raulcd May 12, 2023
bbfb7db

Set executable name to python


raulcd May 12, 2023
e1d51a8

Better comment code


raulcd May 12, 2023
cf41c23

Change variable name to be consistent with similar ones


raulcd May 12, 2023

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


8 changes: 8 additions & 0 deletions
ci/appveyor-cpp-build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,9 @@ set CMAKE_ARGS=-DARROW_DEPENDENCY_SOURCE=CONDA -DARROW_WITH_BZ2=ON
@rem Enable warnings-as-errors
set ARROW_CXXFLAGS=/WX /MP

@rem Install GCS testbench
call %CD%\ci\scripts\install_gcs_testbench.bat

@rem
@rem Build and test Arrow C++ libraries (including Parquet)
@rem
Expand DownExpand Up@@ -70,6 +73,7 @@ cmake -G "%GENERATOR%" %CMAKE_ARGS% ^
 -DARROW_FLIGHT=%ARROW_BUILD_FLIGHT% ^
 -DARROW_FLIGHT_SQL=%ARROW_BUILD_FLIGHT_SQL% ^
 -DARROW_GANDIVA=%ARROW_BUILD_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=ON ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=ON ^
Expand DownExpand Up@@ -97,6 +101,9 @@ cmake --build . --target install --config Release || exit /B
@rem For ORC C++
set TZDIR=%CONDA_PREFIX%\share\zoneinfo

@rem For finding Python executable for GCS tests
set PYTHON=python

ctest --output-on-failure || exit /B

popd
Expand All@@ -114,6 +121,7 @@ set PYARROW_WITH_ACERO=ON
set PYARROW_WITH_DATASET=ON
set PYARROW_WITH_FLIGHT=%ARROW_BUILD_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_BUILD_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_PARQUET=ON
set PYARROW_WITH_PARQUET_ENCRYPTION=ON
set PYARROW_WITH_S3=%ARROW_S3%
Expand Down

25 changes: 25 additions & 0 deletions
ci/scripts/install_gcs_testbench.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
@rem Licensed to the Apache Software Foundation (ASF) under one
@rem or more contributor license agreements. See the NOTICE file
@rem distributed with this work for additional information
@rem regarding copyright ownership. The ASF licenses this file
@rem to you under the Apache License, Version 2.0 (the
@rem "License"); you may not use this file except in compliance
@rem with the License. You may obtain a copy of the License at
@rem
@rem http://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing,
@rem software distributed under the License is distributed on an
@rem "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@rem KIND, either express or implied. See the License for the
@rem specific language governing permissions and limitations
@rem under the License.

@echo on

set GCS_TESTBENCH_VERSION="v0.36.0"

@REM Install GCS testbench %GCS_TESTBENCH_VERSION%
python -m pip install ^
 "https://github.com/googleapis/storage-testbench/archive/%GCS_TESTBENCH_VERSION%.tar.gz" ^
 || exit /B 1

3 changes: 3 additions & 0 deletions
ci/scripts/python_wheel_windows_build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@ set ARROW_ACERO=ON
set ARROW_DATASET=ON
set ARROW_FLIGHT=ON
set ARROW_GANDIVA=OFF
set ARROW_GCS=ON
set ARROW_HDFS=ON
set ARROW_ORC=OFF
set ARROW_PARQUET=ON
Expand DownExpand Up@@ -73,6 +74,7 @@ cmake ^
 -DARROW_FILESYSTEM=ON ^
 -DARROW_FLIGHT=%ARROW_FLIGHT% ^
 -DARROW_GANDIVA=%ARROW_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=%ARROW_HDFS% ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=%ARROW_MIMALLOC% ^
Expand DownExpand Up@@ -111,6 +113,7 @@ set PYARROW_WITH_ACERO=%ARROW_ACERO%
set PYARROW_WITH_DATASET=%ARROW_DATASET%
set PYARROW_WITH_FLIGHT=%ARROW_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_HDFS=%ARROW_HDFS%
set PYARROW_WITH_ORC=%ARROW_ORC%
set PYARROW_WITH_PARQUET=%ARROW_PARQUET%
Expand Down

5 changes: 5 additions & 0 deletions
ci/scripts/python_wheel_windows_test.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -22,6 +22,7 @@ set PYARROW_TEST_CYTHON=ON
set PYARROW_TEST_DATASET=ON
set PYARROW_TEST_FLIGHT=ON
set PYARROW_TEST_GANDIVA=OFF
set PYARROW_TEST_GCS=ON
set PYARROW_TEST_HDFS=ON
set PYARROW_TEST_ORC=OFF
set PYARROW_TEST_PARQUET=ON
Expand All@@ -38,11 +39,15 @@ set PARQUET_TEST_DATA=C:\arrow\submodules\parquet-testing\data
@REM Install testing dependencies
pip install -r C:\arrow\python\requirements-wheel-test.txt || exit /B 1

@REM Install GCS testbench
call "C:\arrow\ci\scripts\install_gcs_testbench.bat"

@REM Install the built wheels
python -m pip install --no-index --find-links=C:\arrow\python\dist\ pyarrow || exit /B 1 

@REM Test that the modules are importable
python -c "import pyarrow" || exit /B 1
python -c "import pyarrow._gcsfs" || exit /B 1
python -c "import pyarrow._hdfs" || exit /B 1 
python -c "import pyarrow._s3fs" || exit /B 1
python -c "import pyarrow.csv" || exit /B 1
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-35193: [Python][Packaging] Enable GCS on Windows wheels#35255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

assignUser
merged 8 commits into

from

May 16, 2023
 Merged


GH-35193: [Python][Packaging] Enable GCS on Windows wheels
#35255
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
3d1dd94

WIP: GH-35193: [Python][Packaging] Enable GCS on Windows wheel


raulcd Apr 20, 2023
b2b891a

Try installing GCS testbench on Windows to run tests


raulcd Apr 20, 2023
50efe4e

Pin urllib3


raulcd May 11, 2023
30bad90

Upgrade testbench to v0.36.0


raulcd May 11, 2023
7466029

Try to add PYTHON environment variable to point to valid python execu…


raulcd May 12, 2023
bbfb7db

Set executable name to python


raulcd May 12, 2023
e1d51a8

Better comment code


raulcd May 12, 2023
cf41c23

Change variable name to be consistent with similar ones


raulcd May 12, 2023

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


8 changes: 8 additions & 0 deletions
ci/appveyor-cpp-build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,9 @@ set CMAKE_ARGS=-DARROW_DEPENDENCY_SOURCE=CONDA -DARROW_WITH_BZ2=ON
@rem Enable warnings-as-errors
set ARROW_CXXFLAGS=/WX /MP

@rem Install GCS testbench
call %CD%\ci\scripts\install_gcs_testbench.bat

@rem
@rem Build and test Arrow C++ libraries (including Parquet)
@rem
Expand DownExpand Up@@ -70,6 +73,7 @@ cmake -G "%GENERATOR%" %CMAKE_ARGS% ^
 -DARROW_FLIGHT=%ARROW_BUILD_FLIGHT% ^
 -DARROW_FLIGHT_SQL=%ARROW_BUILD_FLIGHT_SQL% ^
 -DARROW_GANDIVA=%ARROW_BUILD_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=ON ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=ON ^
Expand DownExpand Up@@ -97,6 +101,9 @@ cmake --build . --target install --config Release || exit /B
@rem For ORC C++
set TZDIR=%CONDA_PREFIX%\share\zoneinfo

@rem For finding Python executable for GCS tests
set PYTHON=python

ctest --output-on-failure || exit /B

popd
Expand All@@ -114,6 +121,7 @@ set PYARROW_WITH_ACERO=ON
set PYARROW_WITH_DATASET=ON
set PYARROW_WITH_FLIGHT=%ARROW_BUILD_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_BUILD_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_PARQUET=ON
set PYARROW_WITH_PARQUET_ENCRYPTION=ON
set PYARROW_WITH_S3=%ARROW_S3%
Expand Down

25 changes: 25 additions & 0 deletions
ci/scripts/install_gcs_testbench.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
@rem Licensed to the Apache Software Foundation (ASF) under one
@rem or more contributor license agreements. See the NOTICE file
@rem distributed with this work for additional information
@rem regarding copyright ownership. The ASF licenses this file
@rem to you under the Apache License, Version 2.0 (the
@rem "License"); you may not use this file except in compliance
@rem with the License. You may obtain a copy of the License at
@rem
@rem http://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing,
@rem software distributed under the License is distributed on an
@rem "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@rem KIND, either express or implied. See the License for the
@rem specific language governing permissions and limitations
@rem under the License.

@echo on

set GCS_TESTBENCH_VERSION="v0.36.0"

@REM Install GCS testbench %GCS_TESTBENCH_VERSION%
python -m pip install ^
 "https://github.com/googleapis/storage-testbench/archive/%GCS_TESTBENCH_VERSION%.tar.gz" ^
 || exit /B 1

3 changes: 3 additions & 0 deletions
ci/scripts/python_wheel_windows_build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@ set ARROW_ACERO=ON
set ARROW_DATASET=ON
set ARROW_FLIGHT=ON
set ARROW_GANDIVA=OFF
set ARROW_GCS=ON
set ARROW_HDFS=ON
set ARROW_ORC=OFF
set ARROW_PARQUET=ON
Expand DownExpand Up@@ -73,6 +74,7 @@ cmake ^
 -DARROW_FILESYSTEM=ON ^
 -DARROW_FLIGHT=%ARROW_FLIGHT% ^
 -DARROW_GANDIVA=%ARROW_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=%ARROW_HDFS% ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=%ARROW_MIMALLOC% ^
Expand DownExpand Up@@ -111,6 +113,7 @@ set PYARROW_WITH_ACERO=%ARROW_ACERO%
set PYARROW_WITH_DATASET=%ARROW_DATASET%
set PYARROW_WITH_FLIGHT=%ARROW_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_HDFS=%ARROW_HDFS%
set PYARROW_WITH_ORC=%ARROW_ORC%
set PYARROW_WITH_PARQUET=%ARROW_PARQUET%
Expand Down

5 changes: 5 additions & 0 deletions
ci/scripts/python_wheel_windows_test.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -22,6 +22,7 @@ set PYARROW_TEST_CYTHON=ON
set PYARROW_TEST_DATASET=ON
set PYARROW_TEST_FLIGHT=ON
set PYARROW_TEST_GANDIVA=OFF
set PYARROW_TEST_GCS=ON
set PYARROW_TEST_HDFS=ON
set PYARROW_TEST_ORC=OFF
set PYARROW_TEST_PARQUET=ON
Expand All@@ -38,11 +39,15 @@ set PARQUET_TEST_DATA=C:\arrow\submodules\parquet-testing\data
@REM Install testing dependencies
pip install -r C:\arrow\python\requirements-wheel-test.txt || exit /B 1

@REM Install GCS testbench
call "C:\arrow\ci\scripts\install_gcs_testbench.bat"

@REM Install the built wheels
python -m pip install --no-index --find-links=C:\arrow\python\dist\ pyarrow || exit /B 1 

@REM Test that the modules are importable
python -c "import pyarrow" || exit /B 1
python -c "import pyarrow._gcsfs" || exit /B 1
python -c "import pyarrow._hdfs" || exit /B 1 
python -c "import pyarrow._s3fs" || exit /B 1
python -c "import pyarrow.csv" || exit /B 1
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
GH-35193: [Python][Packaging] Enable GCS on Windows wheels#35255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

assignUser
merged 8 commits into

from

May 16, 2023
 Merged


GH-35193: [Python][Packaging] Enable GCS on Windows wheels
#35255
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
3d1dd94

WIP: GH-35193: [Python][Packaging] Enable GCS on Windows wheel


raulcd Apr 20, 2023
b2b891a

Try installing GCS testbench on Windows to run tests


raulcd Apr 20, 2023
50efe4e

Pin urllib3


raulcd May 11, 2023
30bad90

Upgrade testbench to v0.36.0


raulcd May 11, 2023
7466029

Try to add PYTHON environment variable to point to valid python execu…


raulcd May 12, 2023
bbfb7db

Set executable name to python


raulcd May 12, 2023
e1d51a8

Better comment code


raulcd May 12, 2023
cf41c23

Change variable name to be consistent with similar ones


raulcd May 12, 2023

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


8 changes: 8 additions & 0 deletions
ci/appveyor-cpp-build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,9 @@ set CMAKE_ARGS=-DARROW_DEPENDENCY_SOURCE=CONDA -DARROW_WITH_BZ2=ON
@rem Enable warnings-as-errors
set ARROW_CXXFLAGS=/WX /MP

@rem Install GCS testbench
call %CD%\ci\scripts\install_gcs_testbench.bat

@rem
@rem Build and test Arrow C++ libraries (including Parquet)
@rem
Expand DownExpand Up@@ -70,6 +73,7 @@ cmake -G "%GENERATOR%" %CMAKE_ARGS% ^
 -DARROW_FLIGHT=%ARROW_BUILD_FLIGHT% ^
 -DARROW_FLIGHT_SQL=%ARROW_BUILD_FLIGHT_SQL% ^
 -DARROW_GANDIVA=%ARROW_BUILD_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=ON ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=ON ^
Expand DownExpand Up@@ -97,6 +101,9 @@ cmake --build . --target install --config Release || exit /B
@rem For ORC C++
set TZDIR=%CONDA_PREFIX%\share\zoneinfo

@rem For finding Python executable for GCS tests
set PYTHON=python

ctest --output-on-failure || exit /B

popd
Expand All@@ -114,6 +121,7 @@ set PYARROW_WITH_ACERO=ON
set PYARROW_WITH_DATASET=ON
set PYARROW_WITH_FLIGHT=%ARROW_BUILD_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_BUILD_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_PARQUET=ON
set PYARROW_WITH_PARQUET_ENCRYPTION=ON
set PYARROW_WITH_S3=%ARROW_S3%
Expand Down

25 changes: 25 additions & 0 deletions
ci/scripts/install_gcs_testbench.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
@rem Licensed to the Apache Software Foundation (ASF) under one
@rem or more contributor license agreements. See the NOTICE file
@rem distributed with this work for additional information
@rem regarding copyright ownership. The ASF licenses this file
@rem to you under the Apache License, Version 2.0 (the
@rem "License"); you may not use this file except in compliance
@rem with the License. You may obtain a copy of the License at
@rem
@rem http://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing,
@rem software distributed under the License is distributed on an
@rem "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@rem KIND, either express or implied. See the License for the
@rem specific language governing permissions and limitations
@rem under the License.

@echo on

set GCS_TESTBENCH_VERSION="v0.36.0"

@REM Install GCS testbench %GCS_TESTBENCH_VERSION%
python -m pip install ^
 "https://github.com/googleapis/storage-testbench/archive/%GCS_TESTBENCH_VERSION%.tar.gz" ^
 || exit /B 1

3 changes: 3 additions & 0 deletions
ci/scripts/python_wheel_windows_build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@ set ARROW_ACERO=ON
set ARROW_DATASET=ON
set ARROW_FLIGHT=ON
set ARROW_GANDIVA=OFF
set ARROW_GCS=ON
set ARROW_HDFS=ON
set ARROW_ORC=OFF
set ARROW_PARQUET=ON
Expand DownExpand Up@@ -73,6 +74,7 @@ cmake ^
 -DARROW_FILESYSTEM=ON ^
 -DARROW_FLIGHT=%ARROW_FLIGHT% ^
 -DARROW_GANDIVA=%ARROW_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=%ARROW_HDFS% ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=%ARROW_MIMALLOC% ^
Expand DownExpand Up@@ -111,6 +113,7 @@ set PYARROW_WITH_ACERO=%ARROW_ACERO%
set PYARROW_WITH_DATASET=%ARROW_DATASET%
set PYARROW_WITH_FLIGHT=%ARROW_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_HDFS=%ARROW_HDFS%
set PYARROW_WITH_ORC=%ARROW_ORC%
set PYARROW_WITH_PARQUET=%ARROW_PARQUET%
Expand Down

5 changes: 5 additions & 0 deletions
ci/scripts/python_wheel_windows_test.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -22,6 +22,7 @@ set PYARROW_TEST_CYTHON=ON
set PYARROW_TEST_DATASET=ON
set PYARROW_TEST_FLIGHT=ON
set PYARROW_TEST_GANDIVA=OFF
set PYARROW_TEST_GCS=ON
set PYARROW_TEST_HDFS=ON
set PYARROW_TEST_ORC=OFF
set PYARROW_TEST_PARQUET=ON
Expand All@@ -38,11 +39,15 @@ set PARQUET_TEST_DATA=C:\arrow\submodules\parquet-testing\data
@REM Install testing dependencies
pip install -r C:\arrow\python\requirements-wheel-test.txt || exit /B 1

@REM Install GCS testbench
call "C:\arrow\ci\scripts\install_gcs_testbench.bat"

@REM Install the built wheels
python -m pip install --no-index --find-links=C:\arrow\python\dist\ pyarrow || exit /B 1 

@REM Test that the modules are importable
python -c "import pyarrow" || exit /B 1
python -c "import pyarrow._gcsfs" || exit /B 1
python -c "import pyarrow._hdfs" || exit /B 1 
python -c "import pyarrow._s3fs" || exit /B 1
python -c "import pyarrow.csv" || exit /B 1
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-35193: [Python][Packaging] Enable GCS on Windows wheels#35255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

assignUser
merged 8 commits into

from

May 16, 2023
 Merged


GH-35193: [Python][Packaging] Enable GCS on Windows wheels
#35255
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
3d1dd94

WIP: GH-35193: [Python][Packaging] Enable GCS on Windows wheel


raulcd Apr 20, 2023
b2b891a

Try installing GCS testbench on Windows to run tests


raulcd Apr 20, 2023
50efe4e

Pin urllib3


raulcd May 11, 2023
30bad90

Upgrade testbench to v0.36.0


raulcd May 11, 2023
7466029

Try to add PYTHON environment variable to point to valid python execu…


raulcd May 12, 2023
bbfb7db

Set executable name to python


raulcd May 12, 2023
e1d51a8

Better comment code


raulcd May 12, 2023
cf41c23

Change variable name to be consistent with similar ones


raulcd May 12, 2023

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


8 changes: 8 additions & 0 deletions
ci/appveyor-cpp-build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,9 @@ set CMAKE_ARGS=-DARROW_DEPENDENCY_SOURCE=CONDA -DARROW_WITH_BZ2=ON
@rem Enable warnings-as-errors
set ARROW_CXXFLAGS=/WX /MP

@rem Install GCS testbench
call %CD%\ci\scripts\install_gcs_testbench.bat

@rem
@rem Build and test Arrow C++ libraries (including Parquet)
@rem
Expand DownExpand Up@@ -70,6 +73,7 @@ cmake -G "%GENERATOR%" %CMAKE_ARGS% ^
 -DARROW_FLIGHT=%ARROW_BUILD_FLIGHT% ^
 -DARROW_FLIGHT_SQL=%ARROW_BUILD_FLIGHT_SQL% ^
 -DARROW_GANDIVA=%ARROW_BUILD_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=ON ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=ON ^
Expand DownExpand Up@@ -97,6 +101,9 @@ cmake --build . --target install --config Release || exit /B
@rem For ORC C++
set TZDIR=%CONDA_PREFIX%\share\zoneinfo

@rem For finding Python executable for GCS tests
set PYTHON=python

ctest --output-on-failure || exit /B

popd
Expand All@@ -114,6 +121,7 @@ set PYARROW_WITH_ACERO=ON
set PYARROW_WITH_DATASET=ON
set PYARROW_WITH_FLIGHT=%ARROW_BUILD_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_BUILD_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_PARQUET=ON
set PYARROW_WITH_PARQUET_ENCRYPTION=ON
set PYARROW_WITH_S3=%ARROW_S3%
Expand Down

25 changes: 25 additions & 0 deletions
ci/scripts/install_gcs_testbench.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
@rem Licensed to the Apache Software Foundation (ASF) under one
@rem or more contributor license agreements. See the NOTICE file
@rem distributed with this work for additional information
@rem regarding copyright ownership. The ASF licenses this file
@rem to you under the Apache License, Version 2.0 (the
@rem "License"); you may not use this file except in compliance
@rem with the License. You may obtain a copy of the License at
@rem
@rem http://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing,
@rem software distributed under the License is distributed on an
@rem "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@rem KIND, either express or implied. See the License for the
@rem specific language governing permissions and limitations
@rem under the License.

@echo on

set GCS_TESTBENCH_VERSION="v0.36.0"

@REM Install GCS testbench %GCS_TESTBENCH_VERSION%
python -m pip install ^
 "https://github.com/googleapis/storage-testbench/archive/%GCS_TESTBENCH_VERSION%.tar.gz" ^
 || exit /B 1

3 changes: 3 additions & 0 deletions
ci/scripts/python_wheel_windows_build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@ set ARROW_ACERO=ON
set ARROW_DATASET=ON
set ARROW_FLIGHT=ON
set ARROW_GANDIVA=OFF
set ARROW_GCS=ON
set ARROW_HDFS=ON
set ARROW_ORC=OFF
set ARROW_PARQUET=ON
Expand DownExpand Up@@ -73,6 +74,7 @@ cmake ^
 -DARROW_FILESYSTEM=ON ^
 -DARROW_FLIGHT=%ARROW_FLIGHT% ^
 -DARROW_GANDIVA=%ARROW_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=%ARROW_HDFS% ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=%ARROW_MIMALLOC% ^
Expand DownExpand Up@@ -111,6 +113,7 @@ set PYARROW_WITH_ACERO=%ARROW_ACERO%
set PYARROW_WITH_DATASET=%ARROW_DATASET%
set PYARROW_WITH_FLIGHT=%ARROW_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_HDFS=%ARROW_HDFS%
set PYARROW_WITH_ORC=%ARROW_ORC%
set PYARROW_WITH_PARQUET=%ARROW_PARQUET%
Expand Down

5 changes: 5 additions & 0 deletions
ci/scripts/python_wheel_windows_test.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -22,6 +22,7 @@ set PYARROW_TEST_CYTHON=ON
set PYARROW_TEST_DATASET=ON
set PYARROW_TEST_FLIGHT=ON
set PYARROW_TEST_GANDIVA=OFF
set PYARROW_TEST_GCS=ON
set PYARROW_TEST_HDFS=ON
set PYARROW_TEST_ORC=OFF
set PYARROW_TEST_PARQUET=ON
Expand All@@ -38,11 +39,15 @@ set PARQUET_TEST_DATA=C:\arrow\submodules\parquet-testing\data
@REM Install testing dependencies
pip install -r C:\arrow\python\requirements-wheel-test.txt || exit /B 1

@REM Install GCS testbench
call "C:\arrow\ci\scripts\install_gcs_testbench.bat"

@REM Install the built wheels
python -m pip install --no-index --find-links=C:\arrow\python\dist\ pyarrow || exit /B 1 

@REM Test that the modules are importable
python -c "import pyarrow" || exit /B 1
python -c "import pyarrow._gcsfs" || exit /B 1
python -c "import pyarrow._hdfs" || exit /B 1 
python -c "import pyarrow._s3fs" || exit /B 1
python -c "import pyarrow.csv" || exit /B 1
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-35193: [Python][Packaging] Enable GCS on Windows wheels#35255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

assignUser
merged 8 commits into

from

May 16, 2023
 Merged


GH-35193: [Python][Packaging] Enable GCS on Windows wheels
#35255
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
3d1dd94

WIP: GH-35193: [Python][Packaging] Enable GCS on Windows wheel


raulcd Apr 20, 2023
b2b891a

Try installing GCS testbench on Windows to run tests


raulcd Apr 20, 2023
50efe4e

Pin urllib3


raulcd May 11, 2023
30bad90

Upgrade testbench to v0.36.0


raulcd May 11, 2023
7466029

Try to add PYTHON environment variable to point to valid python execu…


raulcd May 12, 2023
bbfb7db

Set executable name to python


raulcd May 12, 2023
e1d51a8

Better comment code


raulcd May 12, 2023
cf41c23

Change variable name to be consistent with similar ones


raulcd May 12, 2023

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


8 changes: 8 additions & 0 deletions
ci/appveyor-cpp-build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,9 @@ set CMAKE_ARGS=-DARROW_DEPENDENCY_SOURCE=CONDA -DARROW_WITH_BZ2=ON
@rem Enable warnings-as-errors
set ARROW_CXXFLAGS=/WX /MP

@rem Install GCS testbench
call %CD%\ci\scripts\install_gcs_testbench.bat

@rem
@rem Build and test Arrow C++ libraries (including Parquet)
@rem
Expand DownExpand Up@@ -70,6 +73,7 @@ cmake -G "%GENERATOR%" %CMAKE_ARGS% ^
 -DARROW_FLIGHT=%ARROW_BUILD_FLIGHT% ^
 -DARROW_FLIGHT_SQL=%ARROW_BUILD_FLIGHT_SQL% ^
 -DARROW_GANDIVA=%ARROW_BUILD_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=ON ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=ON ^
Expand DownExpand Up@@ -97,6 +101,9 @@ cmake --build . --target install --config Release || exit /B
@rem For ORC C++
set TZDIR=%CONDA_PREFIX%\share\zoneinfo

@rem For finding Python executable for GCS tests
set PYTHON=python

ctest --output-on-failure || exit /B

popd
Expand All@@ -114,6 +121,7 @@ set PYARROW_WITH_ACERO=ON
set PYARROW_WITH_DATASET=ON
set PYARROW_WITH_FLIGHT=%ARROW_BUILD_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_BUILD_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_PARQUET=ON
set PYARROW_WITH_PARQUET_ENCRYPTION=ON
set PYARROW_WITH_S3=%ARROW_S3%
Expand Down

25 changes: 25 additions & 0 deletions
ci/scripts/install_gcs_testbench.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
@rem Licensed to the Apache Software Foundation (ASF) under one
@rem or more contributor license agreements. See the NOTICE file
@rem distributed with this work for additional information
@rem regarding copyright ownership. The ASF licenses this file
@rem to you under the Apache License, Version 2.0 (the
@rem "License"); you may not use this file except in compliance
@rem with the License. You may obtain a copy of the License at
@rem
@rem http://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing,
@rem software distributed under the License is distributed on an
@rem "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@rem KIND, either express or implied. See the License for the
@rem specific language governing permissions and limitations
@rem under the License.

@echo on

set GCS_TESTBENCH_VERSION="v0.36.0"

@REM Install GCS testbench %GCS_TESTBENCH_VERSION%
python -m pip install ^
 "https://github.com/googleapis/storage-testbench/archive/%GCS_TESTBENCH_VERSION%.tar.gz" ^
 || exit /B 1

3 changes: 3 additions & 0 deletions
ci/scripts/python_wheel_windows_build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@ set ARROW_ACERO=ON
set ARROW_DATASET=ON
set ARROW_FLIGHT=ON
set ARROW_GANDIVA=OFF
set ARROW_GCS=ON
set ARROW_HDFS=ON
set ARROW_ORC=OFF
set ARROW_PARQUET=ON
Expand DownExpand Up@@ -73,6 +74,7 @@ cmake ^
 -DARROW_FILESYSTEM=ON ^
 -DARROW_FLIGHT=%ARROW_FLIGHT% ^
 -DARROW_GANDIVA=%ARROW_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=%ARROW_HDFS% ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=%ARROW_MIMALLOC% ^
Expand DownExpand Up@@ -111,6 +113,7 @@ set PYARROW_WITH_ACERO=%ARROW_ACERO%
set PYARROW_WITH_DATASET=%ARROW_DATASET%
set PYARROW_WITH_FLIGHT=%ARROW_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_HDFS=%ARROW_HDFS%
set PYARROW_WITH_ORC=%ARROW_ORC%
set PYARROW_WITH_PARQUET=%ARROW_PARQUET%
Expand Down

5 changes: 5 additions & 0 deletions
ci/scripts/python_wheel_windows_test.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -22,6 +22,7 @@ set PYARROW_TEST_CYTHON=ON
set PYARROW_TEST_DATASET=ON
set PYARROW_TEST_FLIGHT=ON
set PYARROW_TEST_GANDIVA=OFF
set PYARROW_TEST_GCS=ON
set PYARROW_TEST_HDFS=ON
set PYARROW_TEST_ORC=OFF
set PYARROW_TEST_PARQUET=ON
Expand All@@ -38,11 +39,15 @@ set PARQUET_TEST_DATA=C:\arrow\submodules\parquet-testing\data
@REM Install testing dependencies
pip install -r C:\arrow\python\requirements-wheel-test.txt || exit /B 1

@REM Install GCS testbench
call "C:\arrow\ci\scripts\install_gcs_testbench.bat"

@REM Install the built wheels
python -m pip install --no-index --find-links=C:\arrow\python\dist\ pyarrow || exit /B 1 

@REM Test that the modules are importable
python -c "import pyarrow" || exit /B 1
python -c "import pyarrow._gcsfs" || exit /B 1
python -c "import pyarrow._hdfs" || exit /B 1 
python -c "import pyarrow._s3fs" || exit /B 1
python -c "import pyarrow.csv" || exit /B 1
Expand Down

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
GH-35193: [Python][Packaging] Enable GCS on Windows wheels#35255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

assignUser
merged 8 commits into

from

May 16, 2023
 Merged


GH-35193: [Python][Packaging] Enable GCS on Windows wheels
#35255
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


8 commits


Select commit
Hold shift + click to select a range
3d1dd94

WIP: GH-35193: [Python][Packaging] Enable GCS on Windows wheel


raulcd Apr 20, 2023
b2b891a

Try installing GCS testbench on Windows to run tests


raulcd Apr 20, 2023
50efe4e

Pin urllib3


raulcd May 11, 2023
30bad90

Upgrade testbench to v0.36.0


raulcd May 11, 2023
7466029

Try to add PYTHON environment variable to point to valid python execu…


raulcd May 12, 2023
bbfb7db

Set executable name to python


raulcd May 12, 2023
e1d51a8

Better comment code


raulcd May 12, 2023
cf41c23

Change variable name to be consistent with similar ones


raulcd May 12, 2023

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


8 changes: 8 additions & 0 deletions
ci/appveyor-cpp-build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -42,6 +42,9 @@ set CMAKE_ARGS=-DARROW_DEPENDENCY_SOURCE=CONDA -DARROW_WITH_BZ2=ON
@rem Enable warnings-as-errors
set ARROW_CXXFLAGS=/WX /MP

@rem Install GCS testbench
call %CD%\ci\scripts\install_gcs_testbench.bat

@rem
@rem Build and test Arrow C++ libraries (including Parquet)
@rem
Expand DownExpand Up@@ -70,6 +73,7 @@ cmake -G "%GENERATOR%" %CMAKE_ARGS% ^
 -DARROW_FLIGHT=%ARROW_BUILD_FLIGHT% ^
 -DARROW_FLIGHT_SQL=%ARROW_BUILD_FLIGHT_SQL% ^
 -DARROW_GANDIVA=%ARROW_BUILD_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=ON ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=ON ^
Expand DownExpand Up@@ -97,6 +101,9 @@ cmake --build . --target install --config Release || exit /B
@rem For ORC C++
set TZDIR=%CONDA_PREFIX%\share\zoneinfo

@rem For finding Python executable for GCS tests
set PYTHON=python

ctest --output-on-failure || exit /B

popd
Expand All@@ -114,6 +121,7 @@ set PYARROW_WITH_ACERO=ON
set PYARROW_WITH_DATASET=ON
set PYARROW_WITH_FLIGHT=%ARROW_BUILD_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_BUILD_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_PARQUET=ON
set PYARROW_WITH_PARQUET_ENCRYPTION=ON
set PYARROW_WITH_S3=%ARROW_S3%
Expand Down

25 changes: 25 additions & 0 deletions
ci/scripts/install_gcs_testbench.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
@rem Licensed to the Apache Software Foundation (ASF) under one
@rem or more contributor license agreements. See the NOTICE file
@rem distributed with this work for additional information
@rem regarding copyright ownership. The ASF licenses this file
@rem to you under the Apache License, Version 2.0 (the
@rem "License"); you may not use this file except in compliance
@rem with the License. You may obtain a copy of the License at
@rem
@rem http://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing,
@rem software distributed under the License is distributed on an
@rem "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@rem KIND, either express or implied. See the License for the
@rem specific language governing permissions and limitations
@rem under the License.

@echo on

set GCS_TESTBENCH_VERSION="v0.36.0"

@REM Install GCS testbench %GCS_TESTBENCH_VERSION%
python -m pip install ^
 "https://github.com/googleapis/storage-testbench/archive/%GCS_TESTBENCH_VERSION%.tar.gz" ^
 || exit /B 1

3 changes: 3 additions & 0 deletions
ci/scripts/python_wheel_windows_build.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,7 @@ set ARROW_ACERO=ON
set ARROW_DATASET=ON
set ARROW_FLIGHT=ON
set ARROW_GANDIVA=OFF
set ARROW_GCS=ON
set ARROW_HDFS=ON
set ARROW_ORC=OFF
set ARROW_PARQUET=ON
Expand DownExpand Up@@ -73,6 +74,7 @@ cmake ^
 -DARROW_FILESYSTEM=ON ^
 -DARROW_FLIGHT=%ARROW_FLIGHT% ^
 -DARROW_GANDIVA=%ARROW_GANDIVA% ^
 -DARROW_GCS=%ARROW_GCS% ^
 -DARROW_HDFS=%ARROW_HDFS% ^
 -DARROW_JSON=ON ^
 -DARROW_MIMALLOC=%ARROW_MIMALLOC% ^
Expand DownExpand Up@@ -111,6 +113,7 @@ set PYARROW_WITH_ACERO=%ARROW_ACERO%
set PYARROW_WITH_DATASET=%ARROW_DATASET%
set PYARROW_WITH_FLIGHT=%ARROW_FLIGHT%
set PYARROW_WITH_GANDIVA=%ARROW_GANDIVA%
set PYARROW_WITH_GCS=%ARROW_GCS%
set PYARROW_WITH_HDFS=%ARROW_HDFS%
set PYARROW_WITH_ORC=%ARROW_ORC%
set PYARROW_WITH_PARQUET=%ARROW_PARQUET%
Expand Down

5 changes: 5 additions & 0 deletions
ci/scripts/python_wheel_windows_test.bat

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -22,6 +22,7 @@ set PYARROW_TEST_CYTHON=ON
set PYARROW_TEST_DATASET=ON
set PYARROW_TEST_FLIGHT=ON
set PYARROW_TEST_GANDIVA=OFF
set PYARROW_TEST_GCS=ON
set PYARROW_TEST_HDFS=ON
set PYARROW_TEST_ORC=OFF
set PYARROW_TEST_PARQUET=ON
Expand All@@ -38,11 +39,15 @@ set PARQUET_TEST_DATA=C:\arrow\submodules\parquet-testing\data
@REM Install testing dependencies
pip install -r C:\arrow\python\requirements-wheel-test.txt || exit /B 1

@REM Install GCS testbench
call "C:\arrow\ci\scripts\install_gcs_testbench.bat"

@REM Install the built wheels
python -m pip install --no-index --find-links=C:\arrow\python\dist\ pyarrow || exit /B 1 

@REM Test that the modules are importable
python -c "import pyarrow" || exit /B 1
python -c "import pyarrow._gcsfs" || exit /B 1
python -c "import pyarrow._hdfs" || exit /B 1 
python -c "import pyarrow._s3fs" || exit /B 1
python -c "import pyarrow.csv" || exit /B 1
Expand Down