Skip to content

ARROW-2436: [Rust] Add windows CI - #1949

Closed
paddyhoran wants to merge 19 commits into
apache:masterfrom
paddyhoran:ARROW-2436
Closed

ARROW-2436: [Rust] Add windows CI#1949
paddyhoran wants to merge 19 commits into
apache:masterfrom
paddyhoran:ARROW-2436

Conversation

@paddyhoran

Copy link
Copy Markdown
Contributor

No description provided.

@paddyhoranpaddyhoran changed the title ARROW-2436: [WIP] [Rust] Add windows CIARROW-2436: [Rust] Add windows CIApr 26, 2018
@paddyhoran

Copy link
Copy Markdown
ContributorAuthor

@xhochy some of the travis builds are failing but looking at other PR's it seems that they are failing because the integration tests are being triggered (i.e. other PR's don't fail because they don't trigger the integration tests). I don't believe I have changed anything that should impact the Node.js and openjdk8 builds.

Can you confirm that these integration tests normally pass?

@pitrou

Copy link
Copy Markdown
Member

@paddyhoran We're having problems with npm on Travis-CI currently (see ARROW-2509). You're probably not responsible for them :-)

Comment threadappveyor.yml Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does clcache interfere with the Rust build?

@paddyhoranpaddyhoranApr 26, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just for line 94 where clcache is called (this fails). I guess I could have used NOT "%Rust_Stable%" but the presence of the USE_CLCACHE variable made me think that there might be a valid use case to turn it off for non Rust builds

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine, I just wanted to know.

Comment threadappveyor.yml Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the "%JOB%" to "%BUILD_SCRIPT%" renaming? This looks a bit arbitrary.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The NOT "%JOB%"=="Cmake_Script_Tests" needs to be changed and from what I have read online the best way to do an AND is to use nested if statements. However, is seems AppVeyor does not play nice with multi-line if statements.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the problem is the impossibility of multi-line statements, we should move all this to dedicated scripts (e.g. ci/appveyor-install.bat and ci/appveyor-build.bat).

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, will do. Thanks

Comment threadappveyor.yml Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know if it's possible to have multi-line IF blocks instead?

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above

Comment threadappveyor.yml Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point we will want to selectively disable some matrix builds based on the modified files, as we already do for Travis-CI. The ci/travis_detect_changes.py script does that, it can be extended for AppVeyor. The following logic (from CPython) is supposed to be able to detect changed files on AppVeyor PRs:
https://github.com/python/cpython/blob/master/.github/appveyor.yml#L12-L26

Builds already pile up quickly due to concurrency limitations on AppVeyor, ideally we should not make things worse.

@paddyhoranpaddyhoranApr 26, 2018

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, it's pretty annoying that all cpp builds would re-trigger on changes to the PR. I'll create a JIRA for it. It's probably something I could take a look at.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@paddyhoran

paddyhoran commented Apr 27, 2018

Copy link
Copy Markdown
ContributorAuthor

@pitrou I have re-configured the scripts as you suggested. However, I have some trivial issue that is causing an error. I am polluting the build queue trying to figure out why it is not working. If you have a chance could you take a quick look? You might spot something straight away that I am doing wrong...

If it looks like it should work I'll keep researching...

Thanks

`clcache` is not being found. Checking to see if this is the reason.
@pitrou

Copy link
Copy Markdown
Member

Congrats for sorting this out :-) Is this PR basically ready now?

@paddyhoran

Copy link
Copy Markdown
ContributorAuthor

Hey @pitrou.

Yea, I think this is ready. The only thing I would like to clean up is this (in appveyor-install.bat):

set "PATH=C:\Miniconda36-x64;C:\Miniconda36-x64\Scripts;C:\Miniconda36-x64\Library\bin;%PATH%"

For some reason the following is causing an error:

set MINICONDA=C:\Miniconda36-x64
set "PATH=%MINICONDA%;%MINICONDA%\Scripts;%MINICONDA%\Library\bin;%PATH%"

Another oddity of windows .bat scripting...

For now I'd like to get this merged. I'll circle back to this at some point when I have time to research it more and I'll clean it up.

Thanks very much for all your help!

@pitroupitrou closed this in a2aba52May 1, 2018
@pitrou

Copy link
Copy Markdown
Member

Thanks @paddyhoran !

@paddyhoran
paddyhoran deleted the ARROW-2436 branch May 29, 2018 00:55
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@paddyhoran@pitrou