GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs - #14832

Merged
kou merged 25 commits into
apache:masterfrom
h-vetinari:conda_ci
Dec 12, 2022
Merged

GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs#14832
kou merged 25 commits into
apache:masterfrom
h-vetinari:conda_ci

Conversation

@h-vetinari

@h-vetinarih-vetinari commented Dec 4, 2022•
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor

Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this summary. I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.

It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in tasks.yml).

Finally, it adapts to conda making a long-planned switch w.r.t. to the format / extension of the artefacts it produces.

I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @xhochy
(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).

Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with sparse.

@github-actions

github-actionsBot commented Dec 4, 2022

Copy link
Copy Markdown

@github-actions

github-actionsBot commented Dec 4, 2022

Copy link
Copy Markdown

⚠️ GitHub issue #14828has been automatically assigned in GitHub to PR creator.

@github-actions

github-actionsBot commented Dec 4, 2022

Copy link
Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.

@h-vetinari

h-vetinari commented Dec 4, 2022

Copy link
Copy Markdown
ContributorAuthor

I have a tentative PR for r-arrow, but it's going to need some more work: conda-forge/r-arrow-feedstock#58

@koukou changed the title GH-14828: Fix Conda CIGH-14828: [CI][Conda] Fix nightly jobsDec 5, 2022
@github-actions

github-actionsBot commented Dec 5, 2022

Copy link
Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.

@kou

kou commented Dec 5, 2022

Copy link
Copy Markdown
Member

@github-actions crossbow submit -g conda

@github-actions

github-actionsBot commented Dec 5, 2022

Copy link
Copy Markdown

Revision: a123df8

Submitted crossbow builds: ursacomputing/crossbow @ actions-219630b6c6

TaskStatus
conda-cleanAzure
conda-linux-gcc-openssl1.1.1-aarch64Azure
conda-linux-gcc-openssl1.1.1-cpuAzure
conda-linux-gcc-openssl1.1.1-cudaAzure
conda-linux-gcc-openssl1.1.1-ppc64leAzure
conda-linux-gcc-openssl3-aarch64Azure
conda-linux-gcc-openssl3-cpuAzure
conda-linux-gcc-openssl3-cudaAzure
conda-linux-gcc-openssl3-ppc64leAzure
conda-linux-gcc-py37-cpu-r40Azure
conda-linux-gcc-py37-cpu-r41Azure
conda-osx-arm64-clang-openssl1.1.1Azure
conda-osx-arm64-clang-openssl3Azure
conda-osx-clang-openssl1.1.1Azure
conda-osx-clang-openssl3Azure
conda-osx-clang-py37-r40Azure
conda-osx-clang-py37-r41Azure
conda-win-vs2019-openssl1.1.1Azure
conda-win-vs2019-openssl1.1.1-cudaAzure
conda-win-vs2019-openssl3Azure
conda-win-vs2019-openssl3-cudaAzure
conda-win-vs2019-py37-r40Azure
conda-win-vs2019-py37-r41Azure

@h-vetinari

h-vetinari commented Dec 5, 2022•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

OK, I think I cracked the r-issues in conda-forge/r-arrow-feedstock#58, and updated things correspondingly here.

Also:

  • avoid patch (line-ending issues when applying on windows), patch gcs_server fixture directly see 759e705
  • add r-jobs for linux-aarch64 & osx-arm64 (these were in fact broken by 8139cdd and need ${R_ARGS} to pass in cross-compilation)
  • homogenize naming of conda jobs, see 39587c3

@h-vetinari
consistently sort conda tasks: platform-arch-{cpu,cuda}-<extra>
39587c3
compilers are redundant as they are fixed per platform
@h-vetinari

h-vetinari commented Dec 5, 2022

Copy link
Copy Markdown
ContributorAuthor

@github-actions crossbow submit -g conda

@github-actions

github-actionsBot commented Dec 5, 2022

Copy link
Copy Markdown

Revision: 39587c3

Submitted crossbow builds: ursacomputing/crossbow @ actions-9d7ac78bae

TaskStatus
conda-cleanAzure
conda-linux-aarch64-cpu-openssl1.1.1Azure
conda-linux-aarch64-cpu-openssl3Azure
conda-linux-aarch64-cpu-r41Azure
conda-linux-aarch64-cpu-r42Azure
conda-linux-ppc64le-cpu-openssl1.1.1Azure
conda-linux-ppc64le-cpu-openssl3Azure
conda-linux-x64-cpu-openssl1.1.1Azure
conda-linux-x64-cpu-openssl3Azure
conda-linux-x64-cpu-r41Azure
conda-linux-x64-cpu-r42Azure
conda-linux-x64-cuda-openssl1.1.1Azure
conda-linux-x64-cuda-openssl3Azure
conda-osx-arm64-cpu-r41Azure
conda-osx-arm64-cpu-r42Azure
conda-osx-arm64-openssl1.1.1Azure
conda-osx-arm64-openssl3Azure
conda-osx-x64-cpu-r41Azure
conda-osx-x64-cpu-r42Azure
conda-osx-x64-openssl1.1.1Azure
conda-osx-x64-openssl3Azure
conda-win-x64-cpu-openssl1.1.1Azure
conda-win-x64-cpu-openssl3Azure
conda-win-x64-cpu-r41Azure
conda-win-x64-cuda-openssl1.1.1Azure
conda-win-x64-cuda-openssl3Azure

@h-vetinari

h-vetinari commented Dec 5, 2022

Copy link
Copy Markdown
ContributorAuthor

@github-actions crossbow submit -g conda

h-vetinari added a commit to h-vetinari/r-arrow-feedstock that referenced this pull request Dec 6, 2022
@h-vetinari
sync back changes fromapache/arrow#14832
fed9e16
@h-vetinari

h-vetinari commented Dec 6, 2022

Copy link
Copy Markdown
ContributorAuthor

Any further help you would need from my side?

Ah one more thing, you could help with conda-forge/conda-forge-pinning-feedstock#3793, please.

@h-vetinarih-vetinari mentioned this pull request Dec 7, 2022
Closed
jorisvandenbossche
jorisvandenbossche approved these changes Dec 7, 2022
View reviewed changes

@jorisvandenbosschejorisvandenbossche left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming the changes to the conda feedstock files are just adapted from upstream (as we would like them to be, without unnecessary changes on our side), I didn't review that part, but the changes to pyarrow itself look fine.

@jorisvandenbossche

jorisvandenbossche commented Dec 7, 2022

Copy link
Copy Markdown
Member

Looking at the two remaining failures, one question though: the failures happen on a build with "-r41" suffix (conda-linux-aarch64-cpu-r41 and conda-win-x64-cpu-r41), but it are actually pyarrow test failures (not R arrow). But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?

@h-vetinari

h-vetinari commented Dec 7, 2022•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?

That's unavoidable without some surgery or splitting the feedstocks into libarrow and pyarrow - R only needs the former, but both get built by the same recipe.

I can imagine trying to add a skip for the python bits based on some environment variable (e.g. if we detect R_CONFIG), but so far I've left the feedstock recipes untouched as much as possible (this is also how things were previously, with the difference that only one parrow version got built in the R jobs)

@h-vetinari

h-vetinari commented Dec 7, 2022

Copy link
Copy Markdown
ContributorAuthor

Alternatively, I guess an argument could be made that one job per arch (counting cpu/cuda) could be enough, which builds all python versions, and then latest R on top.

xhochy pushed a commit to conda-forge/r-arrow-feedstock that referenced this pull request Dec 7, 2022
@h-vetinariconda-forge-webservices[bot]
Rebuild for libarrow 10.0.1 (#58)
494d0e6
* build for arrow 10.0.1
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.02.15.22.41
* sync with upstream; be explicit about -std=c++17
* re-add ${R_ARGS}
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.04.19.22.35
* sync back changes from apache/arrow#14832
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
@raulcd

raulcd commented Dec 7, 2022

Copy link
Copy Markdown
Member

Looking at the two remaining failures

I have manually re-run conda-linux-aarch64-cpu-r41 and it has succeeded

raulcd
raulcd approved these changes Dec 7, 2022
View reviewed changes

@raulcdraulcd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried reviewing the different commits individually and looks good to me

@kou

kou commented Dec 7, 2022

Copy link
Copy Markdown
Member

Can we merge this as-is? This fixes our nightly CI.
We can work on some improvements as follow-up tasks.

@kou

kou commented Dec 12, 2022

Copy link
Copy Markdown
Member

No objection. I'll merge this.

@kou
kou merged commit 16d0eb4 into apache:masterDec 12, 2022
@h-vetinari
h-vetinari deleted the conda_ci branch December 12, 2022 21:32
@ursabot

ursabot commented Dec 14, 2022

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 3e2a224 and contender = 16d0eb4. 16d0eb4 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.27% ⬆️0.2%] test-mac-arm
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.0% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 16d0eb4d ec2-t3-xlarge-us-east-2
[Finished] 16d0eb4d test-mac-arm
[Finished] 16d0eb4d ursa-i9-9960x
[Finished] 16d0eb4d ursa-thinkcentre-m75q
[Finished] 3e2a2242 ec2-t3-xlarge-us-east-2
[Finished] 3e2a2242 test-mac-arm
[Finished] 3e2a2242 ursa-i9-9960x
[Finished] 3e2a2242 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

This was referenced Dec 18, 2022
Closed
Open
Closed
Closed
Closed
Closed
This was referenced Mar 21, 2023
Merged
Closed
kou pushed a commit that referenced this pull request Apr 11, 2023
@h-vetinari@assignUser
GH-34669: [Packaging][Conda] Update arrow feedstock dependencies (#34652
e2ddf1c
)
As well as other changes that happened on the feedstock since #14832
* Closes: #34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit to LoongArch-Python/arrow that referenced this pull request May 11, 2023
@h-vetinari@assignUser@liujiacheng777
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
3d732e3
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit to Bit-Quill/arrow that referenced this pull request May 15, 2023
@h-vetinari@assignUser@ArgusLi
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
ed4f80a
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari mentioned this pull request Jul 27, 2023
Closed
@h-vetinarih-vetinari mentioned this pull request Jun 12, 2024
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pitroupitroupitrou left review comments
@koukoukou approved these changes
@raulcdraulcdraulcd approved these changes
@jorisvandenbosschejorisvandenbosschejorisvandenbossche approved these changes

Assignees

No one assigned

Labels

Component: Python

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[CI][Conda] Nightly CI jobs aren't maintained

7 participants

@h-vetinari@kou@pitrou@xhochy@jorisvandenbossche@raulcd@ursabot
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs - #14832
Merged
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Dec 12, 2022
Merged
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs#14832
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Conversation
@h-vetinari
@h-vetinarih-vetinari
commented
Dec 4, 2022•

edited by github-actions
Bot
Loading

Copy link


Copy Markdown

Contributor
Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this summary. I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.
It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in tasks.yml).
Finally, it adapts to conda making a long-planned switch w.r.t. to the format / extension of the artefacts it produces.
I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @xhochy

(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).
Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with sparse.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has been automatically assigned in GitHub to PR creator.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@h-vetinari
h-vetinari
commented
Dec 4, 2022

Copy link


Copy Markdown

ContributorAuthor
I have a tentative PR for r-arrow, but it's going to need some more work: conda-forge/r-arrow-feedstock#58
@koukou
changed the title
GH-14828: Fix Conda CIGH-14828: [CI][Conda] Fix nightly jobsDec 5, 2022
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@kou
kou
commented
Dec 5, 2022

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: a123df8
Submitted crossbow builds: ursacomputing/crossbow @ actions-219630b6c6
TaskStatus
conda-cleanAzure
conda-linux-gcc-openssl1.1.1-aarch64Azure
conda-linux-gcc-openssl1.1.1-cpuAzure
conda-linux-gcc-openssl1.1.1-cudaAzure
conda-linux-gcc-openssl1.1.1-ppc64leAzure
conda-linux-gcc-openssl3-aarch64Azure
conda-linux-gcc-openssl3-cpuAzure
conda-linux-gcc-openssl3-cudaAzure
conda-linux-gcc-openssl3-ppc64leAzure
conda-linux-gcc-py37-cpu-r40Azure
conda-linux-gcc-py37-cpu-r41Azure
conda-osx-arm64-clang-openssl1.1.1Azure
conda-osx-arm64-clang-openssl3Azure
conda-osx-clang-openssl1.1.1Azure
conda-osx-clang-openssl3Azure
conda-osx-clang-py37-r40Azure
conda-osx-clang-py37-r41Azure
conda-win-vs2019-openssl1.1.1Azure
conda-win-vs2019-openssl1.1.1-cudaAzure
conda-win-vs2019-openssl3Azure
conda-win-vs2019-openssl3-cudaAzure
conda-win-vs2019-py37-r40Azure
conda-win-vs2019-py37-r41Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
OK, I think I cracked the r-issues in conda-forge/r-arrow-feedstock#58, and updated things correspondingly here.
Also:
  • avoid patch (line-ending issues when applying on windows), patch gcs_server fixture directly see 759e705
  • add r-jobs for linux-aarch64 & osx-arm64 (these were in fact broken by 8139cdd and need ${R_ARGS} to pass in cross-compilation)
  • homogenize naming of conda jobs, see 39587c3
@h-vetinari
consistently sort conda tasks: platform-arch-{cpu,cuda}-<extra>
39587c3
compilers are redundant as they are fixed per platform
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: 39587c3
Submitted crossbow builds: ursacomputing/crossbow @ actions-9d7ac78bae
TaskStatus
conda-cleanAzure
conda-linux-aarch64-cpu-openssl1.1.1Azure
conda-linux-aarch64-cpu-openssl3Azure
conda-linux-aarch64-cpu-r41Azure
conda-linux-aarch64-cpu-r42Azure
conda-linux-ppc64le-cpu-openssl1.1.1Azure
conda-linux-ppc64le-cpu-openssl3Azure
conda-linux-x64-cpu-openssl1.1.1Azure
conda-linux-x64-cpu-openssl3Azure
conda-linux-x64-cpu-r41Azure
conda-linux-x64-cpu-r42Azure
conda-linux-x64-cuda-openssl1.1.1Azure
conda-linux-x64-cuda-openssl3Azure
conda-osx-arm64-cpu-r41Azure
conda-osx-arm64-cpu-r42Azure
conda-osx-arm64-openssl1.1.1Azure
conda-osx-arm64-openssl3Azure
conda-osx-x64-cpu-r41Azure
conda-osx-x64-cpu-r42Azure
conda-osx-x64-openssl1.1.1Azure
conda-osx-x64-openssl3Azure
conda-win-x64-cpu-openssl1.1.1Azure
conda-win-x64-cpu-openssl3Azure
conda-win-x64-cpu-r41Azure
conda-win-x64-cuda-openssl1.1.1Azure
conda-win-x64-cuda-openssl3Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
h-vetinari added a commit
to h-vetinari/r-arrow-feedstock
that referenced
this pull request
Dec 6, 2022
@h-vetinari
sync back changes fromapache/arrow#14832
fed9e16
@h-vetinari
h-vetinari
commented
Dec 6, 2022

Copy link


Copy Markdown

ContributorAuthor
Any further help you would need from my side?
Ah one more thing, you could help with conda-forge/conda-forge-pinning-feedstock#3793, please.
@h-vetinarih-vetinari
mentioned this pull request
Dec 7, 2022
 Closed

jorisvandenbossche
jorisvandenbossche
approved these changes
Dec 7, 2022
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming the changes to the conda feedstock files are just adapted from upstream (as we would like them to be, without unnecessary changes on our side), I didn't review that part, but the changes to pyarrow itself look fine.
@jorisvandenbossche
jorisvandenbossche
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures, one question though: the failures happen on a build with "-r41" suffix (conda-linux-aarch64-cpu-r41 and conda-win-x64-cpu-r41), but it are actually pyarrow test failures (not R arrow). But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
@h-vetinari
h-vetinari
commented
Dec 7, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
That's unavoidable without some surgery or splitting the feedstocks into libarrow and pyarrow - R only needs the former, but both get built by the same recipe.
I can imagine trying to add a skip for the python bits based on some environment variable (e.g. if we detect R_CONFIG), but so far I've left the feedstock recipes untouched as much as possible (this is also how things were previously, with the difference that only one parrow version got built in the R jobs)
@h-vetinari
h-vetinari
commented
Dec 7, 2022

Copy link


Copy Markdown

ContributorAuthor
Alternatively, I guess an argument could be made that one job per arch (counting cpu/cuda) could be enough, which builds all python versions, and then latest R on top.
xhochy pushed a commit
to conda-forge/r-arrow-feedstock
that referenced
this pull request
Dec 7, 2022
@h-vetinariconda-forge-webservices[bot]
Rebuild for libarrow 10.0.1 (#58)
494d0e6
* build for arrow 10.0.1
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.02.15.22.41
* sync with upstream; be explicit about -std=c++17
* re-add ${R_ARGS}
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.04.19.22.35
* sync back changes from apache/arrow#14832
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
@raulcd
raulcd
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures
I have manually re-run conda-linux-aarch64-cpu-r41 and it has succeeded
raulcd
raulcd
approved these changes
Dec 7, 2022
View reviewed changes
@raulcdraulcd
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried reviewing the different commits individually and looks good to me
@kou
kou
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Can we merge this as-is? This fixes our nightly CI.

We can work on some improvements as follow-up tasks.
@kou
kou
commented
Dec 12, 2022

Copy link


Copy Markdown

Member
No objection. I'll merge this.
@kou
kou
merged commit 16d0eb4
into
apache:masterDec 12, 2022
@h-vetinari
h-vetinari
deleted the

conda_ci

branch
December 12, 2022 21:32
@ursabot
ursabot
commented
Dec 14, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = 3e2a224 and contender = 16d0eb4. 16d0eb4 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Finished ⬇️0.27% ⬆️0.2%] test-mac-arm

[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.0% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 16d0eb4d ec2-t3-xlarge-us-east-2

[Finished] 16d0eb4d test-mac-arm

[Finished] 16d0eb4d ursa-i9-9960x

[Finished] 16d0eb4d ursa-thinkcentre-m75q

[Finished] 3e2a2242 ec2-t3-xlarge-us-east-2

[Finished] 3e2a2242 test-mac-arm

[Finished] 3e2a2242 ursa-i9-9960x

[Finished] 3e2a2242 ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

This was referenced Dec 18, 2022

Closed


Open


Closed


Closed


Closed


Closed


This was referenced Mar 21, 2023
 Merged


Closed

kou pushed a commit
that referenced
this pull request
Apr 11, 2023
@h-vetinari@assignUser
GH-34669: [Packaging][Conda] Update arrow feedstock dependencies (#34652
e2ddf1c
)
As well as other changes that happened on the feedstock since #14832
* Closes: #34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit
to LoongArch-Python/arrow
that referenced
this pull request
May 11, 2023
@h-vetinari@assignUser@liujiacheng777
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
3d732e3
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit
to Bit-Quill/arrow
that referenced
this pull request
May 15, 2023
@h-vetinari@assignUser@ArgusLi
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
ed4f80a
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Jul 27, 2023

Closed

@h-vetinarih-vetinari
mentioned this pull request
Jun 12, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@pitroupitroupitrou left review comments
@koukoukou approved these changes
@raulcdraulcdraulcd approved these changes
@jorisvandenbosschejorisvandenbosschejorisvandenbossche approved these changes

Assignees


No one assigned


Labels

Component: Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[CI][Conda] Nightly CI jobs aren't maintained


7 participants

@h-vetinari@kou@pitrou@xhochy@jorisvandenbossche@raulcd@ursabot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs - #14832
Merged
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Dec 12, 2022
Merged
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs#14832
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Conversation
@h-vetinari
@h-vetinarih-vetinari
commented
Dec 4, 2022•

edited by github-actions
Bot
Loading

Copy link


Copy Markdown

Contributor
Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this summary. I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.
It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in tasks.yml).
Finally, it adapts to conda making a long-planned switch w.r.t. to the format / extension of the artefacts it produces.
I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @xhochy

(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).
Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with sparse.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has been automatically assigned in GitHub to PR creator.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@h-vetinari
h-vetinari
commented
Dec 4, 2022

Copy link


Copy Markdown

ContributorAuthor
I have a tentative PR for r-arrow, but it's going to need some more work: conda-forge/r-arrow-feedstock#58
@koukou
changed the title
GH-14828: Fix Conda CIGH-14828: [CI][Conda] Fix nightly jobsDec 5, 2022
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@kou
kou
commented
Dec 5, 2022

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: a123df8
Submitted crossbow builds: ursacomputing/crossbow @ actions-219630b6c6
TaskStatus
conda-cleanAzure
conda-linux-gcc-openssl1.1.1-aarch64Azure
conda-linux-gcc-openssl1.1.1-cpuAzure
conda-linux-gcc-openssl1.1.1-cudaAzure
conda-linux-gcc-openssl1.1.1-ppc64leAzure
conda-linux-gcc-openssl3-aarch64Azure
conda-linux-gcc-openssl3-cpuAzure
conda-linux-gcc-openssl3-cudaAzure
conda-linux-gcc-openssl3-ppc64leAzure
conda-linux-gcc-py37-cpu-r40Azure
conda-linux-gcc-py37-cpu-r41Azure
conda-osx-arm64-clang-openssl1.1.1Azure
conda-osx-arm64-clang-openssl3Azure
conda-osx-clang-openssl1.1.1Azure
conda-osx-clang-openssl3Azure
conda-osx-clang-py37-r40Azure
conda-osx-clang-py37-r41Azure
conda-win-vs2019-openssl1.1.1Azure
conda-win-vs2019-openssl1.1.1-cudaAzure
conda-win-vs2019-openssl3Azure
conda-win-vs2019-openssl3-cudaAzure
conda-win-vs2019-py37-r40Azure
conda-win-vs2019-py37-r41Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
OK, I think I cracked the r-issues in conda-forge/r-arrow-feedstock#58, and updated things correspondingly here.
Also:
  • avoid patch (line-ending issues when applying on windows), patch gcs_server fixture directly see 759e705
  • add r-jobs for linux-aarch64 & osx-arm64 (these were in fact broken by 8139cdd and need ${R_ARGS} to pass in cross-compilation)
  • homogenize naming of conda jobs, see 39587c3
@h-vetinari
consistently sort conda tasks: platform-arch-{cpu,cuda}-<extra>
39587c3
compilers are redundant as they are fixed per platform
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: 39587c3
Submitted crossbow builds: ursacomputing/crossbow @ actions-9d7ac78bae
TaskStatus
conda-cleanAzure
conda-linux-aarch64-cpu-openssl1.1.1Azure
conda-linux-aarch64-cpu-openssl3Azure
conda-linux-aarch64-cpu-r41Azure
conda-linux-aarch64-cpu-r42Azure
conda-linux-ppc64le-cpu-openssl1.1.1Azure
conda-linux-ppc64le-cpu-openssl3Azure
conda-linux-x64-cpu-openssl1.1.1Azure
conda-linux-x64-cpu-openssl3Azure
conda-linux-x64-cpu-r41Azure
conda-linux-x64-cpu-r42Azure
conda-linux-x64-cuda-openssl1.1.1Azure
conda-linux-x64-cuda-openssl3Azure
conda-osx-arm64-cpu-r41Azure
conda-osx-arm64-cpu-r42Azure
conda-osx-arm64-openssl1.1.1Azure
conda-osx-arm64-openssl3Azure
conda-osx-x64-cpu-r41Azure
conda-osx-x64-cpu-r42Azure
conda-osx-x64-openssl1.1.1Azure
conda-osx-x64-openssl3Azure
conda-win-x64-cpu-openssl1.1.1Azure
conda-win-x64-cpu-openssl3Azure
conda-win-x64-cpu-r41Azure
conda-win-x64-cuda-openssl1.1.1Azure
conda-win-x64-cuda-openssl3Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
h-vetinari added a commit
to h-vetinari/r-arrow-feedstock
that referenced
this pull request
Dec 6, 2022
@h-vetinari
sync back changes fromapache/arrow#14832
fed9e16
@h-vetinari
h-vetinari
commented
Dec 6, 2022

Copy link


Copy Markdown

ContributorAuthor
Any further help you would need from my side?
Ah one more thing, you could help with conda-forge/conda-forge-pinning-feedstock#3793, please.
@h-vetinarih-vetinari
mentioned this pull request
Dec 7, 2022
 Closed

jorisvandenbossche
jorisvandenbossche
approved these changes
Dec 7, 2022
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming the changes to the conda feedstock files are just adapted from upstream (as we would like them to be, without unnecessary changes on our side), I didn't review that part, but the changes to pyarrow itself look fine.
@jorisvandenbossche
jorisvandenbossche
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures, one question though: the failures happen on a build with "-r41" suffix (conda-linux-aarch64-cpu-r41 and conda-win-x64-cpu-r41), but it are actually pyarrow test failures (not R arrow). But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
@h-vetinari
h-vetinari
commented
Dec 7, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
That's unavoidable without some surgery or splitting the feedstocks into libarrow and pyarrow - R only needs the former, but both get built by the same recipe.
I can imagine trying to add a skip for the python bits based on some environment variable (e.g. if we detect R_CONFIG), but so far I've left the feedstock recipes untouched as much as possible (this is also how things were previously, with the difference that only one parrow version got built in the R jobs)
@h-vetinari
h-vetinari
commented
Dec 7, 2022

Copy link


Copy Markdown

ContributorAuthor
Alternatively, I guess an argument could be made that one job per arch (counting cpu/cuda) could be enough, which builds all python versions, and then latest R on top.
xhochy pushed a commit
to conda-forge/r-arrow-feedstock
that referenced
this pull request
Dec 7, 2022
@h-vetinariconda-forge-webservices[bot]
Rebuild for libarrow 10.0.1 (#58)
494d0e6
* build for arrow 10.0.1
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.02.15.22.41
* sync with upstream; be explicit about -std=c++17
* re-add ${R_ARGS}
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.04.19.22.35
* sync back changes from apache/arrow#14832
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
@raulcd
raulcd
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures
I have manually re-run conda-linux-aarch64-cpu-r41 and it has succeeded
raulcd
raulcd
approved these changes
Dec 7, 2022
View reviewed changes
@raulcdraulcd
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried reviewing the different commits individually and looks good to me
@kou
kou
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Can we merge this as-is? This fixes our nightly CI.

We can work on some improvements as follow-up tasks.
@kou
kou
commented
Dec 12, 2022

Copy link


Copy Markdown

Member
No objection. I'll merge this.
@kou
kou
merged commit 16d0eb4
into
apache:masterDec 12, 2022
@h-vetinari
h-vetinari
deleted the

conda_ci

branch
December 12, 2022 21:32
@ursabot
ursabot
commented
Dec 14, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = 3e2a224 and contender = 16d0eb4. 16d0eb4 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Finished ⬇️0.27% ⬆️0.2%] test-mac-arm

[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.0% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 16d0eb4d ec2-t3-xlarge-us-east-2

[Finished] 16d0eb4d test-mac-arm

[Finished] 16d0eb4d ursa-i9-9960x

[Finished] 16d0eb4d ursa-thinkcentre-m75q

[Finished] 3e2a2242 ec2-t3-xlarge-us-east-2

[Finished] 3e2a2242 test-mac-arm

[Finished] 3e2a2242 ursa-i9-9960x

[Finished] 3e2a2242 ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

This was referenced Dec 18, 2022

Closed


Open


Closed


Closed


Closed


Closed


This was referenced Mar 21, 2023
 Merged


Closed

kou pushed a commit
that referenced
this pull request
Apr 11, 2023
@h-vetinari@assignUser
GH-34669: [Packaging][Conda] Update arrow feedstock dependencies (#34652
e2ddf1c
)
As well as other changes that happened on the feedstock since #14832
* Closes: #34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit
to LoongArch-Python/arrow
that referenced
this pull request
May 11, 2023
@h-vetinari@assignUser@liujiacheng777
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
3d732e3
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit
to Bit-Quill/arrow
that referenced
this pull request
May 15, 2023
@h-vetinari@assignUser@ArgusLi
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
ed4f80a
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Jul 27, 2023

Closed

@h-vetinarih-vetinari
mentioned this pull request
Jun 12, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@pitroupitroupitrou left review comments
@koukoukou approved these changes
@raulcdraulcdraulcd approved these changes
@jorisvandenbosschejorisvandenbosschejorisvandenbossche approved these changes

Assignees


No one assigned


Labels

Component: Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[CI][Conda] Nightly CI jobs aren't maintained


7 participants

@h-vetinari@kou@pitrou@xhochy@jorisvandenbossche@raulcd@ursabot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs - #14832
Merged
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Dec 12, 2022
Merged
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs#14832
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Conversation
@h-vetinari
@h-vetinarih-vetinari
commented
Dec 4, 2022•

edited by github-actions
Bot
Loading

Copy link


Copy Markdown

Contributor
Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this summary. I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.
It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in tasks.yml).
Finally, it adapts to conda making a long-planned switch w.r.t. to the format / extension of the artefacts it produces.
I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @xhochy

(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).
Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with sparse.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has been automatically assigned in GitHub to PR creator.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@h-vetinari
h-vetinari
commented
Dec 4, 2022

Copy link


Copy Markdown

ContributorAuthor
I have a tentative PR for r-arrow, but it's going to need some more work: conda-forge/r-arrow-feedstock#58
@koukou
changed the title
GH-14828: Fix Conda CIGH-14828: [CI][Conda] Fix nightly jobsDec 5, 2022
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@kou
kou
commented
Dec 5, 2022

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: a123df8
Submitted crossbow builds: ursacomputing/crossbow @ actions-219630b6c6
TaskStatus
conda-cleanAzure
conda-linux-gcc-openssl1.1.1-aarch64Azure
conda-linux-gcc-openssl1.1.1-cpuAzure
conda-linux-gcc-openssl1.1.1-cudaAzure
conda-linux-gcc-openssl1.1.1-ppc64leAzure
conda-linux-gcc-openssl3-aarch64Azure
conda-linux-gcc-openssl3-cpuAzure
conda-linux-gcc-openssl3-cudaAzure
conda-linux-gcc-openssl3-ppc64leAzure
conda-linux-gcc-py37-cpu-r40Azure
conda-linux-gcc-py37-cpu-r41Azure
conda-osx-arm64-clang-openssl1.1.1Azure
conda-osx-arm64-clang-openssl3Azure
conda-osx-clang-openssl1.1.1Azure
conda-osx-clang-openssl3Azure
conda-osx-clang-py37-r40Azure
conda-osx-clang-py37-r41Azure
conda-win-vs2019-openssl1.1.1Azure
conda-win-vs2019-openssl1.1.1-cudaAzure
conda-win-vs2019-openssl3Azure
conda-win-vs2019-openssl3-cudaAzure
conda-win-vs2019-py37-r40Azure
conda-win-vs2019-py37-r41Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
OK, I think I cracked the r-issues in conda-forge/r-arrow-feedstock#58, and updated things correspondingly here.
Also:
  • avoid patch (line-ending issues when applying on windows), patch gcs_server fixture directly see 759e705
  • add r-jobs for linux-aarch64 & osx-arm64 (these were in fact broken by 8139cdd and need ${R_ARGS} to pass in cross-compilation)
  • homogenize naming of conda jobs, see 39587c3
@h-vetinari
consistently sort conda tasks: platform-arch-{cpu,cuda}-<extra>
39587c3
compilers are redundant as they are fixed per platform
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: 39587c3
Submitted crossbow builds: ursacomputing/crossbow @ actions-9d7ac78bae
TaskStatus
conda-cleanAzure
conda-linux-aarch64-cpu-openssl1.1.1Azure
conda-linux-aarch64-cpu-openssl3Azure
conda-linux-aarch64-cpu-r41Azure
conda-linux-aarch64-cpu-r42Azure
conda-linux-ppc64le-cpu-openssl1.1.1Azure
conda-linux-ppc64le-cpu-openssl3Azure
conda-linux-x64-cpu-openssl1.1.1Azure
conda-linux-x64-cpu-openssl3Azure
conda-linux-x64-cpu-r41Azure
conda-linux-x64-cpu-r42Azure
conda-linux-x64-cuda-openssl1.1.1Azure
conda-linux-x64-cuda-openssl3Azure
conda-osx-arm64-cpu-r41Azure
conda-osx-arm64-cpu-r42Azure
conda-osx-arm64-openssl1.1.1Azure
conda-osx-arm64-openssl3Azure
conda-osx-x64-cpu-r41Azure
conda-osx-x64-cpu-r42Azure
conda-osx-x64-openssl1.1.1Azure
conda-osx-x64-openssl3Azure
conda-win-x64-cpu-openssl1.1.1Azure
conda-win-x64-cpu-openssl3Azure
conda-win-x64-cpu-r41Azure
conda-win-x64-cuda-openssl1.1.1Azure
conda-win-x64-cuda-openssl3Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
h-vetinari added a commit
to h-vetinari/r-arrow-feedstock
that referenced
this pull request
Dec 6, 2022
@h-vetinari
sync back changes fromapache/arrow#14832
fed9e16
@h-vetinari
h-vetinari
commented
Dec 6, 2022

Copy link


Copy Markdown

ContributorAuthor
Any further help you would need from my side?
Ah one more thing, you could help with conda-forge/conda-forge-pinning-feedstock#3793, please.
@h-vetinarih-vetinari
mentioned this pull request
Dec 7, 2022
 Closed

jorisvandenbossche
jorisvandenbossche
approved these changes
Dec 7, 2022
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming the changes to the conda feedstock files are just adapted from upstream (as we would like them to be, without unnecessary changes on our side), I didn't review that part, but the changes to pyarrow itself look fine.
@jorisvandenbossche
jorisvandenbossche
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures, one question though: the failures happen on a build with "-r41" suffix (conda-linux-aarch64-cpu-r41 and conda-win-x64-cpu-r41), but it are actually pyarrow test failures (not R arrow). But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
@h-vetinari
h-vetinari
commented
Dec 7, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
That's unavoidable without some surgery or splitting the feedstocks into libarrow and pyarrow - R only needs the former, but both get built by the same recipe.
I can imagine trying to add a skip for the python bits based on some environment variable (e.g. if we detect R_CONFIG), but so far I've left the feedstock recipes untouched as much as possible (this is also how things were previously, with the difference that only one parrow version got built in the R jobs)
@h-vetinari
h-vetinari
commented
Dec 7, 2022

Copy link


Copy Markdown

ContributorAuthor
Alternatively, I guess an argument could be made that one job per arch (counting cpu/cuda) could be enough, which builds all python versions, and then latest R on top.
xhochy pushed a commit
to conda-forge/r-arrow-feedstock
that referenced
this pull request
Dec 7, 2022
@h-vetinariconda-forge-webservices[bot]
Rebuild for libarrow 10.0.1 (#58)
494d0e6
* build for arrow 10.0.1
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.02.15.22.41
* sync with upstream; be explicit about -std=c++17
* re-add ${R_ARGS}
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.04.19.22.35
* sync back changes from apache/arrow#14832
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
@raulcd
raulcd
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures
I have manually re-run conda-linux-aarch64-cpu-r41 and it has succeeded
raulcd
raulcd
approved these changes
Dec 7, 2022
View reviewed changes
@raulcdraulcd
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried reviewing the different commits individually and looks good to me
@kou
kou
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Can we merge this as-is? This fixes our nightly CI.

We can work on some improvements as follow-up tasks.
@kou
kou
commented
Dec 12, 2022

Copy link


Copy Markdown

Member
No objection. I'll merge this.
@kou
kou
merged commit 16d0eb4
into
apache:masterDec 12, 2022
@h-vetinari
h-vetinari
deleted the

conda_ci

branch
December 12, 2022 21:32
@ursabot
ursabot
commented
Dec 14, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = 3e2a224 and contender = 16d0eb4. 16d0eb4 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Finished ⬇️0.27% ⬆️0.2%] test-mac-arm

[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.0% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 16d0eb4d ec2-t3-xlarge-us-east-2

[Finished] 16d0eb4d test-mac-arm

[Finished] 16d0eb4d ursa-i9-9960x

[Finished] 16d0eb4d ursa-thinkcentre-m75q

[Finished] 3e2a2242 ec2-t3-xlarge-us-east-2

[Finished] 3e2a2242 test-mac-arm

[Finished] 3e2a2242 ursa-i9-9960x

[Finished] 3e2a2242 ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

This was referenced Dec 18, 2022

Closed


Open


Closed


Closed


Closed


Closed


This was referenced Mar 21, 2023
 Merged


Closed

kou pushed a commit
that referenced
this pull request
Apr 11, 2023
@h-vetinari@assignUser
GH-34669: [Packaging][Conda] Update arrow feedstock dependencies (#34652
e2ddf1c
)
As well as other changes that happened on the feedstock since #14832
* Closes: #34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit
to LoongArch-Python/arrow
that referenced
this pull request
May 11, 2023
@h-vetinari@assignUser@liujiacheng777
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
3d732e3
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit
to Bit-Quill/arrow
that referenced
this pull request
May 15, 2023
@h-vetinari@assignUser@ArgusLi
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
ed4f80a
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Jul 27, 2023

Closed

@h-vetinarih-vetinari
mentioned this pull request
Jun 12, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@pitroupitroupitrou left review comments
@koukoukou approved these changes
@raulcdraulcdraulcd approved these changes
@jorisvandenbosschejorisvandenbosschejorisvandenbossche approved these changes

Assignees


No one assigned


Labels

Component: Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[CI][Conda] Nightly CI jobs aren't maintained


7 participants

@h-vetinari@kou@pitrou@xhochy@jorisvandenbossche@raulcd@ursabot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs - #14832
Merged
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Dec 12, 2022
Merged
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs#14832
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Conversation
@h-vetinari
@h-vetinarih-vetinari
commented
Dec 4, 2022•

edited by github-actions
Bot
Loading

Copy link


Copy Markdown

Contributor
Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this summary. I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.
It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in tasks.yml).
Finally, it adapts to conda making a long-planned switch w.r.t. to the format / extension of the artefacts it produces.
I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @xhochy

(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).
Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with sparse.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has been automatically assigned in GitHub to PR creator.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@h-vetinari
h-vetinari
commented
Dec 4, 2022

Copy link


Copy Markdown

ContributorAuthor
I have a tentative PR for r-arrow, but it's going to need some more work: conda-forge/r-arrow-feedstock#58
@koukou
changed the title
GH-14828: Fix Conda CIGH-14828: [CI][Conda] Fix nightly jobsDec 5, 2022
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@kou
kou
commented
Dec 5, 2022

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: a123df8
Submitted crossbow builds: ursacomputing/crossbow @ actions-219630b6c6
TaskStatus
conda-cleanAzure
conda-linux-gcc-openssl1.1.1-aarch64Azure
conda-linux-gcc-openssl1.1.1-cpuAzure
conda-linux-gcc-openssl1.1.1-cudaAzure
conda-linux-gcc-openssl1.1.1-ppc64leAzure
conda-linux-gcc-openssl3-aarch64Azure
conda-linux-gcc-openssl3-cpuAzure
conda-linux-gcc-openssl3-cudaAzure
conda-linux-gcc-openssl3-ppc64leAzure
conda-linux-gcc-py37-cpu-r40Azure
conda-linux-gcc-py37-cpu-r41Azure
conda-osx-arm64-clang-openssl1.1.1Azure
conda-osx-arm64-clang-openssl3Azure
conda-osx-clang-openssl1.1.1Azure
conda-osx-clang-openssl3Azure
conda-osx-clang-py37-r40Azure
conda-osx-clang-py37-r41Azure
conda-win-vs2019-openssl1.1.1Azure
conda-win-vs2019-openssl1.1.1-cudaAzure
conda-win-vs2019-openssl3Azure
conda-win-vs2019-openssl3-cudaAzure
conda-win-vs2019-py37-r40Azure
conda-win-vs2019-py37-r41Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
OK, I think I cracked the r-issues in conda-forge/r-arrow-feedstock#58, and updated things correspondingly here.
Also:
  • avoid patch (line-ending issues when applying on windows), patch gcs_server fixture directly see 759e705
  • add r-jobs for linux-aarch64 & osx-arm64 (these were in fact broken by 8139cdd and need ${R_ARGS} to pass in cross-compilation)
  • homogenize naming of conda jobs, see 39587c3
@h-vetinari
consistently sort conda tasks: platform-arch-{cpu,cuda}-<extra>
39587c3
compilers are redundant as they are fixed per platform
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: 39587c3
Submitted crossbow builds: ursacomputing/crossbow @ actions-9d7ac78bae
TaskStatus
conda-cleanAzure
conda-linux-aarch64-cpu-openssl1.1.1Azure
conda-linux-aarch64-cpu-openssl3Azure
conda-linux-aarch64-cpu-r41Azure
conda-linux-aarch64-cpu-r42Azure
conda-linux-ppc64le-cpu-openssl1.1.1Azure
conda-linux-ppc64le-cpu-openssl3Azure
conda-linux-x64-cpu-openssl1.1.1Azure
conda-linux-x64-cpu-openssl3Azure
conda-linux-x64-cpu-r41Azure
conda-linux-x64-cpu-r42Azure
conda-linux-x64-cuda-openssl1.1.1Azure
conda-linux-x64-cuda-openssl3Azure
conda-osx-arm64-cpu-r41Azure
conda-osx-arm64-cpu-r42Azure
conda-osx-arm64-openssl1.1.1Azure
conda-osx-arm64-openssl3Azure
conda-osx-x64-cpu-r41Azure
conda-osx-x64-cpu-r42Azure
conda-osx-x64-openssl1.1.1Azure
conda-osx-x64-openssl3Azure
conda-win-x64-cpu-openssl1.1.1Azure
conda-win-x64-cpu-openssl3Azure
conda-win-x64-cpu-r41Azure
conda-win-x64-cuda-openssl1.1.1Azure
conda-win-x64-cuda-openssl3Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
h-vetinari added a commit
to h-vetinari/r-arrow-feedstock
that referenced
this pull request
Dec 6, 2022
@h-vetinari
sync back changes fromapache/arrow#14832
fed9e16
@h-vetinari
h-vetinari
commented
Dec 6, 2022

Copy link


Copy Markdown

ContributorAuthor
Any further help you would need from my side?
Ah one more thing, you could help with conda-forge/conda-forge-pinning-feedstock#3793, please.
@h-vetinarih-vetinari
mentioned this pull request
Dec 7, 2022
 Closed

jorisvandenbossche
jorisvandenbossche
approved these changes
Dec 7, 2022
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming the changes to the conda feedstock files are just adapted from upstream (as we would like them to be, without unnecessary changes on our side), I didn't review that part, but the changes to pyarrow itself look fine.
@jorisvandenbossche
jorisvandenbossche
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures, one question though: the failures happen on a build with "-r41" suffix (conda-linux-aarch64-cpu-r41 and conda-win-x64-cpu-r41), but it are actually pyarrow test failures (not R arrow). But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
@h-vetinari
h-vetinari
commented
Dec 7, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
That's unavoidable without some surgery or splitting the feedstocks into libarrow and pyarrow - R only needs the former, but both get built by the same recipe.
I can imagine trying to add a skip for the python bits based on some environment variable (e.g. if we detect R_CONFIG), but so far I've left the feedstock recipes untouched as much as possible (this is also how things were previously, with the difference that only one parrow version got built in the R jobs)
@h-vetinari
h-vetinari
commented
Dec 7, 2022

Copy link


Copy Markdown

ContributorAuthor
Alternatively, I guess an argument could be made that one job per arch (counting cpu/cuda) could be enough, which builds all python versions, and then latest R on top.
xhochy pushed a commit
to conda-forge/r-arrow-feedstock
that referenced
this pull request
Dec 7, 2022
@h-vetinariconda-forge-webservices[bot]
Rebuild for libarrow 10.0.1 (#58)
494d0e6
* build for arrow 10.0.1
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.02.15.22.41
* sync with upstream; be explicit about -std=c++17
* re-add ${R_ARGS}
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.04.19.22.35
* sync back changes from apache/arrow#14832
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
@raulcd
raulcd
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures
I have manually re-run conda-linux-aarch64-cpu-r41 and it has succeeded
raulcd
raulcd
approved these changes
Dec 7, 2022
View reviewed changes
@raulcdraulcd
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried reviewing the different commits individually and looks good to me
@kou
kou
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Can we merge this as-is? This fixes our nightly CI.

We can work on some improvements as follow-up tasks.
@kou
kou
commented
Dec 12, 2022

Copy link


Copy Markdown

Member
No objection. I'll merge this.
@kou
kou
merged commit 16d0eb4
into
apache:masterDec 12, 2022
@h-vetinari
h-vetinari
deleted the

conda_ci

branch
December 12, 2022 21:32
@ursabot
ursabot
commented
Dec 14, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = 3e2a224 and contender = 16d0eb4. 16d0eb4 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Finished ⬇️0.27% ⬆️0.2%] test-mac-arm

[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.0% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 16d0eb4d ec2-t3-xlarge-us-east-2

[Finished] 16d0eb4d test-mac-arm

[Finished] 16d0eb4d ursa-i9-9960x

[Finished] 16d0eb4d ursa-thinkcentre-m75q

[Finished] 3e2a2242 ec2-t3-xlarge-us-east-2

[Finished] 3e2a2242 test-mac-arm

[Finished] 3e2a2242 ursa-i9-9960x

[Finished] 3e2a2242 ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

This was referenced Dec 18, 2022

Closed


Open


Closed


Closed


Closed


Closed


This was referenced Mar 21, 2023
 Merged


Closed

kou pushed a commit
that referenced
this pull request
Apr 11, 2023
@h-vetinari@assignUser
GH-34669: [Packaging][Conda] Update arrow feedstock dependencies (#34652
e2ddf1c
)
As well as other changes that happened on the feedstock since #14832
* Closes: #34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit
to LoongArch-Python/arrow
that referenced
this pull request
May 11, 2023
@h-vetinari@assignUser@liujiacheng777
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
3d732e3
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit
to Bit-Quill/arrow
that referenced
this pull request
May 15, 2023
@h-vetinari@assignUser@ArgusLi
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
ed4f80a
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Jul 27, 2023

Closed

@h-vetinarih-vetinari
mentioned this pull request
Jun 12, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@pitroupitroupitrou left review comments
@koukoukou approved these changes
@raulcdraulcdraulcd approved these changes
@jorisvandenbosschejorisvandenbosschejorisvandenbossche approved these changes

Assignees


No one assigned


Labels

Component: Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[CI][Conda] Nightly CI jobs aren't maintained


7 participants

@h-vetinari@kou@pitrou@xhochy@jorisvandenbossche@raulcd@ursabot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs - #14832
Merged
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Dec 12, 2022
Merged
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs#14832
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Conversation
@h-vetinari
@h-vetinarih-vetinari
commented
Dec 4, 2022•

edited by github-actions
Bot
Loading

Copy link


Copy Markdown

Contributor
Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this summary. I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.
It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in tasks.yml).
Finally, it adapts to conda making a long-planned switch w.r.t. to the format / extension of the artefacts it produces.
I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @xhochy

(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).
Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with sparse.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has been automatically assigned in GitHub to PR creator.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@h-vetinari
h-vetinari
commented
Dec 4, 2022

Copy link


Copy Markdown

ContributorAuthor
I have a tentative PR for r-arrow, but it's going to need some more work: conda-forge/r-arrow-feedstock#58
@koukou
changed the title
GH-14828: Fix Conda CIGH-14828: [CI][Conda] Fix nightly jobsDec 5, 2022
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@kou
kou
commented
Dec 5, 2022

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: a123df8
Submitted crossbow builds: ursacomputing/crossbow @ actions-219630b6c6
TaskStatus
conda-cleanAzure
conda-linux-gcc-openssl1.1.1-aarch64Azure
conda-linux-gcc-openssl1.1.1-cpuAzure
conda-linux-gcc-openssl1.1.1-cudaAzure
conda-linux-gcc-openssl1.1.1-ppc64leAzure
conda-linux-gcc-openssl3-aarch64Azure
conda-linux-gcc-openssl3-cpuAzure
conda-linux-gcc-openssl3-cudaAzure
conda-linux-gcc-openssl3-ppc64leAzure
conda-linux-gcc-py37-cpu-r40Azure
conda-linux-gcc-py37-cpu-r41Azure
conda-osx-arm64-clang-openssl1.1.1Azure
conda-osx-arm64-clang-openssl3Azure
conda-osx-clang-openssl1.1.1Azure
conda-osx-clang-openssl3Azure
conda-osx-clang-py37-r40Azure
conda-osx-clang-py37-r41Azure
conda-win-vs2019-openssl1.1.1Azure
conda-win-vs2019-openssl1.1.1-cudaAzure
conda-win-vs2019-openssl3Azure
conda-win-vs2019-openssl3-cudaAzure
conda-win-vs2019-py37-r40Azure
conda-win-vs2019-py37-r41Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
OK, I think I cracked the r-issues in conda-forge/r-arrow-feedstock#58, and updated things correspondingly here.
Also:
  • avoid patch (line-ending issues when applying on windows), patch gcs_server fixture directly see 759e705
  • add r-jobs for linux-aarch64 & osx-arm64 (these were in fact broken by 8139cdd and need ${R_ARGS} to pass in cross-compilation)
  • homogenize naming of conda jobs, see 39587c3
@h-vetinari
consistently sort conda tasks: platform-arch-{cpu,cuda}-<extra>
39587c3
compilers are redundant as they are fixed per platform
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: 39587c3
Submitted crossbow builds: ursacomputing/crossbow @ actions-9d7ac78bae
TaskStatus
conda-cleanAzure
conda-linux-aarch64-cpu-openssl1.1.1Azure
conda-linux-aarch64-cpu-openssl3Azure
conda-linux-aarch64-cpu-r41Azure
conda-linux-aarch64-cpu-r42Azure
conda-linux-ppc64le-cpu-openssl1.1.1Azure
conda-linux-ppc64le-cpu-openssl3Azure
conda-linux-x64-cpu-openssl1.1.1Azure
conda-linux-x64-cpu-openssl3Azure
conda-linux-x64-cpu-r41Azure
conda-linux-x64-cpu-r42Azure
conda-linux-x64-cuda-openssl1.1.1Azure
conda-linux-x64-cuda-openssl3Azure
conda-osx-arm64-cpu-r41Azure
conda-osx-arm64-cpu-r42Azure
conda-osx-arm64-openssl1.1.1Azure
conda-osx-arm64-openssl3Azure
conda-osx-x64-cpu-r41Azure
conda-osx-x64-cpu-r42Azure
conda-osx-x64-openssl1.1.1Azure
conda-osx-x64-openssl3Azure
conda-win-x64-cpu-openssl1.1.1Azure
conda-win-x64-cpu-openssl3Azure
conda-win-x64-cpu-r41Azure
conda-win-x64-cuda-openssl1.1.1Azure
conda-win-x64-cuda-openssl3Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
h-vetinari added a commit
to h-vetinari/r-arrow-feedstock
that referenced
this pull request
Dec 6, 2022
@h-vetinari
sync back changes fromapache/arrow#14832
fed9e16
@h-vetinari
h-vetinari
commented
Dec 6, 2022

Copy link


Copy Markdown

ContributorAuthor
Any further help you would need from my side?
Ah one more thing, you could help with conda-forge/conda-forge-pinning-feedstock#3793, please.
@h-vetinarih-vetinari
mentioned this pull request
Dec 7, 2022
 Closed

jorisvandenbossche
jorisvandenbossche
approved these changes
Dec 7, 2022
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming the changes to the conda feedstock files are just adapted from upstream (as we would like them to be, without unnecessary changes on our side), I didn't review that part, but the changes to pyarrow itself look fine.
@jorisvandenbossche
jorisvandenbossche
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures, one question though: the failures happen on a build with "-r41" suffix (conda-linux-aarch64-cpu-r41 and conda-win-x64-cpu-r41), but it are actually pyarrow test failures (not R arrow). But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
@h-vetinari
h-vetinari
commented
Dec 7, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
That's unavoidable without some surgery or splitting the feedstocks into libarrow and pyarrow - R only needs the former, but both get built by the same recipe.
I can imagine trying to add a skip for the python bits based on some environment variable (e.g. if we detect R_CONFIG), but so far I've left the feedstock recipes untouched as much as possible (this is also how things were previously, with the difference that only one parrow version got built in the R jobs)
@h-vetinari
h-vetinari
commented
Dec 7, 2022

Copy link


Copy Markdown

ContributorAuthor
Alternatively, I guess an argument could be made that one job per arch (counting cpu/cuda) could be enough, which builds all python versions, and then latest R on top.
xhochy pushed a commit
to conda-forge/r-arrow-feedstock
that referenced
this pull request
Dec 7, 2022
@h-vetinariconda-forge-webservices[bot]
Rebuild for libarrow 10.0.1 (#58)
494d0e6
* build for arrow 10.0.1
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.02.15.22.41
* sync with upstream; be explicit about -std=c++17
* re-add ${R_ARGS}
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.04.19.22.35
* sync back changes from apache/arrow#14832
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
@raulcd
raulcd
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures
I have manually re-run conda-linux-aarch64-cpu-r41 and it has succeeded
raulcd
raulcd
approved these changes
Dec 7, 2022
View reviewed changes
@raulcdraulcd
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried reviewing the different commits individually and looks good to me
@kou
kou
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Can we merge this as-is? This fixes our nightly CI.

We can work on some improvements as follow-up tasks.
@kou
kou
commented
Dec 12, 2022

Copy link


Copy Markdown

Member
No objection. I'll merge this.
@kou
kou
merged commit 16d0eb4
into
apache:masterDec 12, 2022
@h-vetinari
h-vetinari
deleted the

conda_ci

branch
December 12, 2022 21:32
@ursabot
ursabot
commented
Dec 14, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = 3e2a224 and contender = 16d0eb4. 16d0eb4 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Finished ⬇️0.27% ⬆️0.2%] test-mac-arm

[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.0% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 16d0eb4d ec2-t3-xlarge-us-east-2

[Finished] 16d0eb4d test-mac-arm

[Finished] 16d0eb4d ursa-i9-9960x

[Finished] 16d0eb4d ursa-thinkcentre-m75q

[Finished] 3e2a2242 ec2-t3-xlarge-us-east-2

[Finished] 3e2a2242 test-mac-arm

[Finished] 3e2a2242 ursa-i9-9960x

[Finished] 3e2a2242 ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

This was referenced Dec 18, 2022

Closed


Open


Closed


Closed


Closed


Closed


This was referenced Mar 21, 2023
 Merged


Closed

kou pushed a commit
that referenced
this pull request
Apr 11, 2023
@h-vetinari@assignUser
GH-34669: [Packaging][Conda] Update arrow feedstock dependencies (#34652
e2ddf1c
)
As well as other changes that happened on the feedstock since #14832
* Closes: #34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit
to LoongArch-Python/arrow
that referenced
this pull request
May 11, 2023
@h-vetinari@assignUser@liujiacheng777
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
3d732e3
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit
to Bit-Quill/arrow
that referenced
this pull request
May 15, 2023
@h-vetinari@assignUser@ArgusLi
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
ed4f80a
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Jul 27, 2023

Closed

@h-vetinarih-vetinari
mentioned this pull request
Jun 12, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@pitroupitroupitrou left review comments
@koukoukou approved these changes
@raulcdraulcdraulcd approved these changes
@jorisvandenbosschejorisvandenbosschejorisvandenbossche approved these changes

Assignees


No one assigned


Labels

Component: Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[CI][Conda] Nightly CI jobs aren't maintained


7 participants

@h-vetinari@kou@pitrou@xhochy@jorisvandenbossche@raulcd@ursabot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs - #14832
Merged
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Dec 12, 2022
Merged
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs#14832
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Conversation
@h-vetinari
@h-vetinarih-vetinari
commented
Dec 4, 2022•

edited by github-actions
Bot
Loading

Copy link


Copy Markdown

Contributor
Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this summary. I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.
It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in tasks.yml).
Finally, it adapts to conda making a long-planned switch w.r.t. to the format / extension of the artefacts it produces.
I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @xhochy

(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).
Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with sparse.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has been automatically assigned in GitHub to PR creator.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@h-vetinari
h-vetinari
commented
Dec 4, 2022

Copy link


Copy Markdown

ContributorAuthor
I have a tentative PR for r-arrow, but it's going to need some more work: conda-forge/r-arrow-feedstock#58
@koukou
changed the title
GH-14828: Fix Conda CIGH-14828: [CI][Conda] Fix nightly jobsDec 5, 2022
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@kou
kou
commented
Dec 5, 2022

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: a123df8
Submitted crossbow builds: ursacomputing/crossbow @ actions-219630b6c6
TaskStatus
conda-cleanAzure
conda-linux-gcc-openssl1.1.1-aarch64Azure
conda-linux-gcc-openssl1.1.1-cpuAzure
conda-linux-gcc-openssl1.1.1-cudaAzure
conda-linux-gcc-openssl1.1.1-ppc64leAzure
conda-linux-gcc-openssl3-aarch64Azure
conda-linux-gcc-openssl3-cpuAzure
conda-linux-gcc-openssl3-cudaAzure
conda-linux-gcc-openssl3-ppc64leAzure
conda-linux-gcc-py37-cpu-r40Azure
conda-linux-gcc-py37-cpu-r41Azure
conda-osx-arm64-clang-openssl1.1.1Azure
conda-osx-arm64-clang-openssl3Azure
conda-osx-clang-openssl1.1.1Azure
conda-osx-clang-openssl3Azure
conda-osx-clang-py37-r40Azure
conda-osx-clang-py37-r41Azure
conda-win-vs2019-openssl1.1.1Azure
conda-win-vs2019-openssl1.1.1-cudaAzure
conda-win-vs2019-openssl3Azure
conda-win-vs2019-openssl3-cudaAzure
conda-win-vs2019-py37-r40Azure
conda-win-vs2019-py37-r41Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
OK, I think I cracked the r-issues in conda-forge/r-arrow-feedstock#58, and updated things correspondingly here.
Also:
  • avoid patch (line-ending issues when applying on windows), patch gcs_server fixture directly see 759e705
  • add r-jobs for linux-aarch64 & osx-arm64 (these were in fact broken by 8139cdd and need ${R_ARGS} to pass in cross-compilation)
  • homogenize naming of conda jobs, see 39587c3
@h-vetinari
consistently sort conda tasks: platform-arch-{cpu,cuda}-<extra>
39587c3
compilers are redundant as they are fixed per platform
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: 39587c3
Submitted crossbow builds: ursacomputing/crossbow @ actions-9d7ac78bae
TaskStatus
conda-cleanAzure
conda-linux-aarch64-cpu-openssl1.1.1Azure
conda-linux-aarch64-cpu-openssl3Azure
conda-linux-aarch64-cpu-r41Azure
conda-linux-aarch64-cpu-r42Azure
conda-linux-ppc64le-cpu-openssl1.1.1Azure
conda-linux-ppc64le-cpu-openssl3Azure
conda-linux-x64-cpu-openssl1.1.1Azure
conda-linux-x64-cpu-openssl3Azure
conda-linux-x64-cpu-r41Azure
conda-linux-x64-cpu-r42Azure
conda-linux-x64-cuda-openssl1.1.1Azure
conda-linux-x64-cuda-openssl3Azure
conda-osx-arm64-cpu-r41Azure
conda-osx-arm64-cpu-r42Azure
conda-osx-arm64-openssl1.1.1Azure
conda-osx-arm64-openssl3Azure
conda-osx-x64-cpu-r41Azure
conda-osx-x64-cpu-r42Azure
conda-osx-x64-openssl1.1.1Azure
conda-osx-x64-openssl3Azure
conda-win-x64-cpu-openssl1.1.1Azure
conda-win-x64-cpu-openssl3Azure
conda-win-x64-cpu-r41Azure
conda-win-x64-cuda-openssl1.1.1Azure
conda-win-x64-cuda-openssl3Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
h-vetinari added a commit
to h-vetinari/r-arrow-feedstock
that referenced
this pull request
Dec 6, 2022
@h-vetinari
sync back changes fromapache/arrow#14832
fed9e16
@h-vetinari
h-vetinari
commented
Dec 6, 2022

Copy link


Copy Markdown

ContributorAuthor
Any further help you would need from my side?
Ah one more thing, you could help with conda-forge/conda-forge-pinning-feedstock#3793, please.
@h-vetinarih-vetinari
mentioned this pull request
Dec 7, 2022
 Closed

jorisvandenbossche
jorisvandenbossche
approved these changes
Dec 7, 2022
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming the changes to the conda feedstock files are just adapted from upstream (as we would like them to be, without unnecessary changes on our side), I didn't review that part, but the changes to pyarrow itself look fine.
@jorisvandenbossche
jorisvandenbossche
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures, one question though: the failures happen on a build with "-r41" suffix (conda-linux-aarch64-cpu-r41 and conda-win-x64-cpu-r41), but it are actually pyarrow test failures (not R arrow). But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
@h-vetinari
h-vetinari
commented
Dec 7, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
That's unavoidable without some surgery or splitting the feedstocks into libarrow and pyarrow - R only needs the former, but both get built by the same recipe.
I can imagine trying to add a skip for the python bits based on some environment variable (e.g. if we detect R_CONFIG), but so far I've left the feedstock recipes untouched as much as possible (this is also how things were previously, with the difference that only one parrow version got built in the R jobs)
@h-vetinari
h-vetinari
commented
Dec 7, 2022

Copy link


Copy Markdown

ContributorAuthor
Alternatively, I guess an argument could be made that one job per arch (counting cpu/cuda) could be enough, which builds all python versions, and then latest R on top.
xhochy pushed a commit
to conda-forge/r-arrow-feedstock
that referenced
this pull request
Dec 7, 2022
@h-vetinariconda-forge-webservices[bot]
Rebuild for libarrow 10.0.1 (#58)
494d0e6
* build for arrow 10.0.1
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.02.15.22.41
* sync with upstream; be explicit about -std=c++17
* re-add ${R_ARGS}
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.04.19.22.35
* sync back changes from apache/arrow#14832
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
@raulcd
raulcd
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures
I have manually re-run conda-linux-aarch64-cpu-r41 and it has succeeded
raulcd
raulcd
approved these changes
Dec 7, 2022
View reviewed changes
@raulcdraulcd
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried reviewing the different commits individually and looks good to me
@kou
kou
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Can we merge this as-is? This fixes our nightly CI.

We can work on some improvements as follow-up tasks.
@kou
kou
commented
Dec 12, 2022

Copy link


Copy Markdown

Member
No objection. I'll merge this.
@kou
kou
merged commit 16d0eb4
into
apache:masterDec 12, 2022
@h-vetinari
h-vetinari
deleted the

conda_ci

branch
December 12, 2022 21:32
@ursabot
ursabot
commented
Dec 14, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = 3e2a224 and contender = 16d0eb4. 16d0eb4 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Finished ⬇️0.27% ⬆️0.2%] test-mac-arm

[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.0% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 16d0eb4d ec2-t3-xlarge-us-east-2

[Finished] 16d0eb4d test-mac-arm

[Finished] 16d0eb4d ursa-i9-9960x

[Finished] 16d0eb4d ursa-thinkcentre-m75q

[Finished] 3e2a2242 ec2-t3-xlarge-us-east-2

[Finished] 3e2a2242 test-mac-arm

[Finished] 3e2a2242 ursa-i9-9960x

[Finished] 3e2a2242 ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

This was referenced Dec 18, 2022

Closed


Open


Closed


Closed


Closed


Closed


This was referenced Mar 21, 2023
 Merged


Closed

kou pushed a commit
that referenced
this pull request
Apr 11, 2023
@h-vetinari@assignUser
GH-34669: [Packaging][Conda] Update arrow feedstock dependencies (#34652
e2ddf1c
)
As well as other changes that happened on the feedstock since #14832
* Closes: #34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit
to LoongArch-Python/arrow
that referenced
this pull request
May 11, 2023
@h-vetinari@assignUser@liujiacheng777
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
3d732e3
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit
to Bit-Quill/arrow
that referenced
this pull request
May 15, 2023
@h-vetinari@assignUser@ArgusLi
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
ed4f80a
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Jul 27, 2023

Closed

@h-vetinarih-vetinari
mentioned this pull request
Jun 12, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@pitroupitroupitrou left review comments
@koukoukou approved these changes
@raulcdraulcdraulcd approved these changes
@jorisvandenbosschejorisvandenbosschejorisvandenbossche approved these changes

Assignees


No one assigned


Labels

Component: Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[CI][Conda] Nightly CI jobs aren't maintained


7 participants

@h-vetinari@kou@pitrou@xhochy@jorisvandenbossche@raulcd@ursabot

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs - #14832
Merged
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Dec 12, 2022
Merged
GH-14828: [CI][Conda] Sync with conda-forge, fix nightly jobs#14832
kou merged 25 commits into
apache:masterfrom 
h-vetinari:conda_ci
Conversation
@h-vetinari
@h-vetinarih-vetinari
commented
Dec 4, 2022•

edited by github-actions
Bot
Loading

Copy link


Copy Markdown

Contributor
Synching after conda-forge/arrow-cpp-feedstock#875, which does quite a lot of things, see this summary. I'm not keeping the commit history here, but it might be instructive to check the commits there to see why certain changes came about.
It also fixes the CI that was broken by a3ef64b (undoing the changes of #14102 in tasks.yml).
Finally, it adapts to conda making a long-planned switch w.r.t. to the format / extension of the artefacts it produces.
I'm very likely going to need some help (or at least pointers) for the R-stuff. CC @xhochy

(for context, I never got a response to conda-forge/r-arrow-feedstock#55, but I'll open a PR to build against libarrow 10).
Once this is done, I can open issues to tackle the tests that shouldn't be failing, resp. the segfaults on PPC resp. in conjunction with sparse.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has been automatically assigned in GitHub to PR creator.
@github-actions
github-actionsBot
commented
Dec 4, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@h-vetinari
h-vetinari
commented
Dec 4, 2022

Copy link


Copy Markdown

ContributorAuthor
I have a tentative PR for r-arrow, but it's going to need some more work: conda-forge/r-arrow-feedstock#58
@koukou
changed the title
GH-14828: Fix Conda CIGH-14828: [CI][Conda] Fix nightly jobsDec 5, 2022
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

⚠️ GitHub issue #14828has no components, please add labels for components.
@kou
kou
commented
Dec 5, 2022

Copy link


Copy Markdown

Member
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: a123df8
Submitted crossbow builds: ursacomputing/crossbow @ actions-219630b6c6
TaskStatus
conda-cleanAzure
conda-linux-gcc-openssl1.1.1-aarch64Azure
conda-linux-gcc-openssl1.1.1-cpuAzure
conda-linux-gcc-openssl1.1.1-cudaAzure
conda-linux-gcc-openssl1.1.1-ppc64leAzure
conda-linux-gcc-openssl3-aarch64Azure
conda-linux-gcc-openssl3-cpuAzure
conda-linux-gcc-openssl3-cudaAzure
conda-linux-gcc-openssl3-ppc64leAzure
conda-linux-gcc-py37-cpu-r40Azure
conda-linux-gcc-py37-cpu-r41Azure
conda-osx-arm64-clang-openssl1.1.1Azure
conda-osx-arm64-clang-openssl3Azure
conda-osx-clang-openssl1.1.1Azure
conda-osx-clang-openssl3Azure
conda-osx-clang-py37-r40Azure
conda-osx-clang-py37-r41Azure
conda-win-vs2019-openssl1.1.1Azure
conda-win-vs2019-openssl1.1.1-cudaAzure
conda-win-vs2019-openssl3Azure
conda-win-vs2019-openssl3-cudaAzure
conda-win-vs2019-py37-r40Azure
conda-win-vs2019-py37-r41Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
OK, I think I cracked the r-issues in conda-forge/r-arrow-feedstock#58, and updated things correspondingly here.
Also:
  • avoid patch (line-ending issues when applying on windows), patch gcs_server fixture directly see 759e705
  • add r-jobs for linux-aarch64 & osx-arm64 (these were in fact broken by 8139cdd and need ${R_ARGS} to pass in cross-compilation)
  • homogenize naming of conda jobs, see 39587c3
@h-vetinari
consistently sort conda tasks: platform-arch-{cpu,cuda}-<extra>
39587c3
compilers are redundant as they are fixed per platform
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
@github-actions
github-actionsBot
commented
Dec 5, 2022

Copy link


Copy Markdown

Revision: 39587c3
Submitted crossbow builds: ursacomputing/crossbow @ actions-9d7ac78bae
TaskStatus
conda-cleanAzure
conda-linux-aarch64-cpu-openssl1.1.1Azure
conda-linux-aarch64-cpu-openssl3Azure
conda-linux-aarch64-cpu-r41Azure
conda-linux-aarch64-cpu-r42Azure
conda-linux-ppc64le-cpu-openssl1.1.1Azure
conda-linux-ppc64le-cpu-openssl3Azure
conda-linux-x64-cpu-openssl1.1.1Azure
conda-linux-x64-cpu-openssl3Azure
conda-linux-x64-cpu-r41Azure
conda-linux-x64-cpu-r42Azure
conda-linux-x64-cuda-openssl1.1.1Azure
conda-linux-x64-cuda-openssl3Azure
conda-osx-arm64-cpu-r41Azure
conda-osx-arm64-cpu-r42Azure
conda-osx-arm64-openssl1.1.1Azure
conda-osx-arm64-openssl3Azure
conda-osx-x64-cpu-r41Azure
conda-osx-x64-cpu-r42Azure
conda-osx-x64-openssl1.1.1Azure
conda-osx-x64-openssl3Azure
conda-win-x64-cpu-openssl1.1.1Azure
conda-win-x64-cpu-openssl3Azure
conda-win-x64-cpu-r41Azure
conda-win-x64-cuda-openssl1.1.1Azure
conda-win-x64-cuda-openssl3Azure
@h-vetinari
h-vetinari
commented
Dec 5, 2022

Copy link


Copy Markdown

ContributorAuthor
@github-actions crossbow submit -g conda
h-vetinari added a commit
to h-vetinari/r-arrow-feedstock
that referenced
this pull request
Dec 6, 2022
@h-vetinari
sync back changes fromapache/arrow#14832
fed9e16
@h-vetinari
h-vetinari
commented
Dec 6, 2022

Copy link


Copy Markdown

ContributorAuthor
Any further help you would need from my side?
Ah one more thing, you could help with conda-forge/conda-forge-pinning-feedstock#3793, please.
@h-vetinarih-vetinari
mentioned this pull request
Dec 7, 2022
 Closed

jorisvandenbossche
jorisvandenbossche
approved these changes
Dec 7, 2022
View reviewed changes
@jorisvandenbosschejorisvandenbossche
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming the changes to the conda feedstock files are just adapted from upstream (as we would like them to be, without unnecessary changes on our side), I didn't review that part, but the changes to pyarrow itself look fine.
@jorisvandenbossche
jorisvandenbossche
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures, one question though: the failures happen on a build with "-r41" suffix (conda-linux-aarch64-cpu-r41 and conda-win-x64-cpu-r41), but it are actually pyarrow test failures (not R arrow). But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
@h-vetinari
h-vetinari
commented
Dec 7, 2022•

edited

Loading

Copy link


Copy Markdown

ContributorAuthor
But there is also a build for the same architecture with "-py3" suffix (which is also running the pyarrow tests). That seems a bit superfluous to run the python tests in each of the R builds as well?
That's unavoidable without some surgery or splitting the feedstocks into libarrow and pyarrow - R only needs the former, but both get built by the same recipe.
I can imagine trying to add a skip for the python bits based on some environment variable (e.g. if we detect R_CONFIG), but so far I've left the feedstock recipes untouched as much as possible (this is also how things were previously, with the difference that only one parrow version got built in the R jobs)
@h-vetinari
h-vetinari
commented
Dec 7, 2022

Copy link


Copy Markdown

ContributorAuthor
Alternatively, I guess an argument could be made that one job per arch (counting cpu/cuda) could be enough, which builds all python versions, and then latest R on top.
xhochy pushed a commit
to conda-forge/r-arrow-feedstock
that referenced
this pull request
Dec 7, 2022
@h-vetinariconda-forge-webservices[bot]
Rebuild for libarrow 10.0.1 (#58)
494d0e6
* build for arrow 10.0.1
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.02.15.22.41
* sync with upstream; be explicit about -std=c++17
* re-add ${R_ARGS}
* MNT: Re-rendered with conda-build 3.23.2, conda-smithy 3.22.0, and conda-forge-pinning 2022.12.04.19.22.35
* sync back changes from apache/arrow#14832
Co-authored-by: conda-forge-webservices[bot] <91080706+conda-forge-webservices[bot]@users.noreply.github.com>
@raulcd
raulcd
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Looking at the two remaining failures
I have manually re-run conda-linux-aarch64-cpu-r41 and it has succeeded
raulcd
raulcd
approved these changes
Dec 7, 2022
View reviewed changes
@raulcdraulcd
left a comment


Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried reviewing the different commits individually and looks good to me
@kou
kou
commented
Dec 7, 2022

Copy link


Copy Markdown

Member
Can we merge this as-is? This fixes our nightly CI.

We can work on some improvements as follow-up tasks.
@kou
kou
commented
Dec 12, 2022

Copy link


Copy Markdown

Member
No objection. I'll merge this.
@kou
kou
merged commit 16d0eb4
into
apache:masterDec 12, 2022
@h-vetinari
h-vetinari
deleted the

conda_ci

branch
December 12, 2022 21:32
@ursabot
ursabot
commented
Dec 14, 2022

Copy link


Copy Markdown

Benchmark runs are scheduled for baseline = 3e2a224 and contender = 16d0eb4. 16d0eb4 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.

Conbench compare runs links:

[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2

[Finished ⬇️0.27% ⬆️0.2%] test-mac-arm

[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x

[Finished ⬇️0.0% ⬆️0.0%] ursa-thinkcentre-m75q

Buildkite builds:

[Finished] 16d0eb4d ec2-t3-xlarge-us-east-2

[Finished] 16d0eb4d test-mac-arm

[Finished] 16d0eb4d ursa-i9-9960x

[Finished] 16d0eb4d ursa-thinkcentre-m75q

[Finished] 3e2a2242 ec2-t3-xlarge-us-east-2

[Finished] 3e2a2242 test-mac-arm

[Finished] 3e2a2242 ursa-i9-9960x

[Finished] 3e2a2242 ursa-thinkcentre-m75q

Supported benchmarks:

ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True

test-mac-arm: Supported benchmark langs: C++, Python, R

ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript

ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

This was referenced Dec 18, 2022

Closed


Open


Closed


Closed


Closed


Closed


This was referenced Mar 21, 2023
 Merged


Closed

kou pushed a commit
that referenced
this pull request
Apr 11, 2023
@h-vetinari@assignUser
GH-34669: [Packaging][Conda] Update arrow feedstock dependencies (#34652
e2ddf1c
)
As well as other changes that happened on the feedstock since #14832
* Closes: #34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
liujiacheng777 pushed a commit
to LoongArch-Python/arrow
that referenced
this pull request
May 11, 2023
@h-vetinari@assignUser@liujiacheng777
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
3d732e3
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
ArgusLi pushed a commit
to Bit-Quill/arrow
that referenced
this pull request
May 15, 2023
@h-vetinari@assignUser@ArgusLi
apacheGH-34669: [Packaging][Conda] Update arrow feedstock dependencies (
ed4f80a
apache#34652)
As well as other changes that happened on the feedstock since apache#14832
* Closes: apache#34669
Lead-authored-by: H. Vetinari <h.vetinari@gmx.com>
Co-authored-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@h-vetinarih-vetinari
mentioned this pull request
Jul 27, 2023

Closed

@h-vetinarih-vetinari
mentioned this pull request
Jun 12, 2024
 Closed

Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@pitroupitroupitrou left review comments
@koukoukou approved these changes
@raulcdraulcdraulcd approved these changes
@jorisvandenbosschejorisvandenbosschejorisvandenbossche approved these changes

Assignees


No one assigned


Labels

Component: Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

[CI][Conda] Nightly CI jobs aren't maintained


7 participants

@h-vetinari@kou@pitrou@xhochy@jorisvandenbossche@raulcd@ursabot