Skip to content

GH-38431: [Python][CI] Update fs.type_name checks for s3fs tests - #38455

Merged
kou merged 2 commits into
apache:mainfrom
AlenkaF:gh-38431-s3fs-name-change
Oct 26, 2023
Merged

GH-38431: [Python][CI] Update fs.type_name checks for s3fs tests#38455
kou merged 2 commits into
apache:mainfrom
AlenkaF:gh-38431-s3fs-name-change

Conversation

@AlenkaF

@AlenkaFAlenkaF commented Oct 25, 2023

Copy link
Copy Markdown
Member

Rationale for this change

Appveyor CI is failing https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/48347810. It seems the reason for the test failures is a change in the type_name (from "py::fsspec+s3" to "py::fsspec+('s3', 's3a')") and due to it tests are not being skipped.

What changes are included in this PR?

Update the check for type_name in case of PyFileSystem(FSSpecHandler(s3fs.S3FileSystem())).

@AlenkaF

Copy link
Copy Markdown
MemberAuthor

AppVeyor is passing but other Python builds are still using the old version of s3fs (2023.9.2). Will change the pin in conda_env_python.txt

@AlenkaF
AlenkaF marked this pull request as ready for review October 25, 2023 13:04
@raulcd

Copy link
Copy Markdown
Member

@github-actions crossbow submit -g python

@github-actions

Copy link
Copy Markdown

Revision: 3a53d11

Submitted crossbow builds: ursacomputing/crossbow @ actions-a83bce7a72

TaskStatus
test-conda-python-3.10Github Actions
test-conda-python-3.10-cython2Github Actions
test-conda-python-3.10-hdfs-2.9.2Github Actions
test-conda-python-3.10-hdfs-3.2.1Github Actions
test-conda-python-3.10-pandas-latestGithub Actions
test-conda-python-3.10-pandas-nightlyGithub Actions
test-conda-python-3.10-spark-v3.5.0Github Actions
test-conda-python-3.10-substraitGithub Actions
test-conda-python-3.11Github Actions
test-conda-python-3.11-dask-latestGithub Actions
test-conda-python-3.11-dask-upstream_develGithub Actions
test-conda-python-3.11-hypothesisGithub Actions
test-conda-python-3.11-pandas-upstream_develGithub Actions
test-conda-python-3.11-spark-masterGithub Actions
test-conda-python-3.12Github Actions
test-conda-python-3.8Github Actions
test-conda-python-3.8-pandas-1.0Github Actions
test-conda-python-3.8-spark-v3.5.0Github Actions
test-conda-python-3.9Github Actions
test-conda-python-3.9-pandas-latestGithub Actions
test-cuda-pythonGithub Actions
test-debian-11-python-3Azure
test-fedora-35-python-3Azure
test-ubuntu-20.04-python-3Azure
test-ubuntu-22.04-python-3Github Actions

@raulcd

Copy link
Copy Markdown
Member

@github-actions crossbow submit -g conda

@github-actions

Copy link
Copy Markdown

Revision: 3a53d11

Submitted crossbow builds: ursacomputing/crossbow @ actions-716f9fca2a

TaskStatus
conda-cleanAzure
conda-linux-aarch64-cpu-py3Azure
conda-linux-aarch64-cpu-r42Azure
conda-linux-aarch64-cpu-r43Azure
conda-linux-aarch64-cuda-py3Azure
conda-linux-ppc64le-cpu-py3Azure
conda-linux-ppc64le-cuda-py3Azure
conda-linux-x64-cpu-py3Azure
conda-linux-x64-cpu-r42Azure
conda-linux-x64-cpu-r43Azure
conda-linux-x64-cuda-py3Azure
conda-osx-arm64-cpu-py3Azure
conda-osx-arm64-cpu-r42Azure
conda-osx-arm64-cpu-r43Azure
conda-osx-x64-cpu-py3Azure
conda-osx-x64-cpu-r42Azure
conda-osx-x64-cpu-r43Azure
conda-win-x64-cpu-py3Azure
conda-win-x64-cpu-r41Azure
conda-win-x64-cuda-py3Azure

kou
kou approved these changes Oct 26, 2023

@koukou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@kou
kou merged commit 3f4aeab into apache:mainOct 26, 2023
@koukou removed the awaiting review Awaiting review label Oct 26, 2023
@github-actionsgithub-actionsBot added the awaiting merge Awaiting merge label Oct 26, 2023
@AlenkaF
AlenkaF deleted the gh-38431-s3fs-name-change branch October 26, 2023 09:53
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 5 benchmarking runs that have been run so far on merge-commit 3f4aeab.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.

raulcd pushed a commit that referenced this pull request Nov 6, 2023
)
### Rationale for this change
Appveyor CI is failing https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/48347810. It seems the reason for the test failures is a change in the `type_name` (from `"py::fsspec+s3"` to `"py::fsspec+('s3', 's3a')"`) and due to it tests are not being skipped.
### What changes are included in this PR?
Update the check for `type_name` in case of `PyFileSystem(FSSpecHandler(s3fs.S3FileSystem()))`.
* Closes: #38431
Authored-by: AlenkaF <frim.alenka@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
loicalleyne pushed a commit to loicalleyne/arrow that referenced this pull request Nov 13, 2023
apache#38455)
### Rationale for this change
Appveyor CI is failing https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/48347810. It seems the reason for the test failures is a change in the `type_name` (from `"py::fsspec+s3"` to `"py::fsspec+('s3', 's3a')"`) and due to it tests are not being skipped.
### What changes are included in this PR?
Update the check for `type_name` in case of `PyFileSystem(FSSpecHandler(s3fs.S3FileSystem()))`.
* Closes: apache#38431
Authored-by: AlenkaF <frim.alenka@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this pull request Feb 19, 2024
apache#38455)
### Rationale for this change
Appveyor CI is failing https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/48347810. It seems the reason for the test failures is a change in the `type_name` (from `"py::fsspec+s3"` to `"py::fsspec+('s3', 's3a')"`) and due to it tests are not being skipped.
### What changes are included in this PR?
Update the check for `type_name` in case of `PyFileSystem(FSSpecHandler(s3fs.S3FileSystem()))`.
* Closes: apache#38431
Authored-by: AlenkaF <frim.alenka@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Python][CI] AppVeyor CI is failing on test_fs.py::test_filesystem_is_functional_after_pickling

3 participants

@AlenkaF@raulcd@kou