rustbuild: Add option for enabling partial LLVM rebuilds by petrochenkov · Pull Request #40329 · rust-lang/rust

petrochenkov · 2017-03-07T20:28:04Z

@alexcrichton , you probably didn't notice my late comment on #40236, but here's an implementation of that suggestion, it supersedes c652a4f.

r? @alexcrichton

alexcrichton · 2017-03-07T22:52:22Z

Oh sorry I did indeed miss that, my bad!

FWIW I'm down for basically any behavior that's reasonable for local development. Local developers are much more forgiving with behavior like "looks weird? eh I'll blow it away". CI is much more unforgiving :(

That being said we're in a much better place than we were when all this infrastructure was created long ago. Almost all CI builders start from scratch unconditionally (e.g. no llvm build dir existing). We then rely on sccache to quickly compile LLVM (as it should all be cached).

Unfortunately, though, sccache doesn't quite work on MSVC with LLVM just yet (effort to get ninja working hasn't completed yet). So to counter this the LLVM build directory is cached across MSVC builds on appveyor. For us though AppVeyor handles the invalidation by not downloading the cache when src/rustllvm/llvm-auto-clean-trigger changes.

So the tl;dr; for CI is that:

All builds except MSVC have completely clean build directories, LLVM is always compiled from scratch
On MSVC, the build dir is cached across builds, and cleaned when src/rustllvm/llvm-auto-clean-trigger changes.

For now that means that we still require src/rustllvm/llvm-auto-clean-trigger to get changed whenever we touch LLVM, but in the future once sccache works well with LLVM on MSVC we can just remove all this "trigger" infrastructure.

(if you're willing to help out with sccache on ninja it'd be much appreciated as well!)

alexcrichton · 2017-03-07T22:53:32Z

This PR seems fine by me, but want to default this option to false? You could then modify src/ci/run.sh to pass a ./configure option to turn it on by default on CI.

petrochenkov · 2017-03-08T13:36:39Z

Updated.

alexcrichton · 2017-03-08T16:03:32Z

@bors: r+

bors · 2017-03-08T16:03:33Z

📌 Commit 5697240 has been approved by alexcrichton

bors · 2017-03-09T10:42:49Z

🔒 Merge conflict

alexcrichton · 2017-03-09T15:04:41Z

@bors: retry

alexcrichton · 2017-03-10T16:18:13Z

@bors: r+

bors · 2017-03-10T16:18:14Z

📌 Commit c4b9284 has been approved by alexcrichton

alexcrichton · 2017-03-10T21:03:09Z

@bors: r+

bors · 2017-03-10T21:03:10Z

📌 Commit 8c5bdd6 has been approved by alexcrichton

Rollup of 38 pull requests - Successful merges: #39202, #39820, #39918, #39921, #40092, #40146, #40199, #40225, #40239, #40257, #40259, #40261, #40277, #40278, #40287, #40297, #40311, #40315, #40319, #40324, #40336, #40340, #40344, #40345, #40367, #40369, #40372, #40373, #40379, #40385, #40386, #40389, #40400, #40404, #40410, #40422, #40423, #40424 - Failed merges: #40220, #40329, #40426

bors · 2017-03-11T08:30:04Z

☔ The latest upstream changes (presumably #40432) made this pull request unmergeable. Please resolve the merge conflicts.

petrochenkov · 2017-03-11T10:16:44Z

@bors r=alexcrichton

bors · 2017-03-11T10:16:45Z

📌 Commit 4cdbe5c has been approved by alexcrichton

bors · 2017-03-12T10:46:29Z

⌛ Testing commit 4cdbe5c with merge 4123c76...

bors · 2017-03-12T10:50:01Z

☔ The latest upstream changes (presumably #40257) made this pull request unmergeable. Please resolve the merge conflicts.

It may not exist, so we don't want to spuriously generate an error.

petrochenkov · 2017-03-12T11:08:41Z

@bors r=alexcrichton

bors · 2017-03-12T11:08:41Z

📌 Commit 4cda4d6 has been approved by alexcrichton

bors · 2017-03-12T13:44:56Z

⌛ Testing commit 4cda4d6 with merge 49ab594...

bors · 2017-03-12T16:46:32Z

💔 Test failed - status-travis

petrochenkov · 2017-03-12T17:13:22Z

Not sure what happened, too much logging from sccache, the part of the log after 4MB is lost.

@bors retry

bors · 2017-03-12T22:22:16Z

⌛ Testing commit 4cda4d6 with merge 4ab92b8...

bors · 2017-03-12T22:35:20Z

💔 Test failed - status-appveyor

petrochenkov · 2017-03-12T22:44:06Z

sccache failure:

error: failed to execute compile
caused by: error reading compile response from server
caused by: IoError(Error { repr: Os { code: 10054, message: "An existing connection was forcibly closed by the remote host." } })
caused by: An existing connection was forcibly closed by the remote host. (os error 10054)

@bors retry

bors · 2017-03-12T23:10:14Z

⌛ Testing commit 4cda4d6 with merge 2936719...

bors · 2017-03-12T23:22:06Z

💔 Test failed - status-appveyor

petrochenkov · 2017-03-13T07:34:34Z

sccache failure

@bors retry

bors · 2017-03-13T08:54:53Z

⌛ Testing commit 4cda4d6 with merge 0a1bce1...

bors · 2017-03-13T09:24:07Z

💔 Test failed - status-travis

alexcrichton · 2017-03-13T14:49:24Z

@bors: retry

spurious sccache failure Spurious failures in sccache #40240

bors · 2017-03-13T15:31:55Z

⌛ Testing commit 4cda4d6 with merge fd182c4...

@alexcrichton

rustbuild: Add option for enabling partial LLVM rebuilds @alexcrichton , you probably didn't notice my [late comment](#40236 (comment)) on #40236, but here's an implementation of that suggestion, it supersedes c652a4f. r? @alexcrichton

bors · 2017-03-13T18:39:34Z

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing fd182c4 to master...

rust-highfive assigned alexcrichton Mar 7, 2017

petrochenkov force-pushed the llreuse branch from c1aab7f to 5697240Compare March 8, 2017 13:36

alexcrichton mentioned this pull request Mar 10, 2017
Rollup of 36 pull requests #40418
Closed

alexcrichton force-pushed the llreuse branch from 5697240 to c4b9284Compare March 10, 2017 16:18

alexcrichton mentioned this pull request Mar 10, 2017
Rollup of 35 pull requests #40420
Closed

alexcrichton mentioned this pull request Mar 10, 2017
Rollup of 38 pull requests #40432
Merged

petrochenkov force-pushed the llreuse branch from 8c5bdd6 to 4cdbe5cCompare March 11, 2017 10:16

petrochenkovand others added 3 commits March 12, 2017 14:02

rustbuild: Add option for enabling partial LLVM rebuilds
9b8b3b2

Default llvm.clean-rebuild to false
362aa9a

Allow cleaning llvm build dir to fail
4cda4d6
It may not exist, so we don't want to spuriously generate an error.

petrochenkov force-pushed the llreuse branch from 4cdbe5c to 4cda4d6Compare March 12, 2017 11:08

alexcrichton mentioned this pull request Mar 13, 2017
Add timeouts to cache lookups and handle server eofs mozilla/sccache#79
Merged

bors merged commit 4cda4d6 into rust-lang:masterMar 13, 2017

petrochenkov deleted the llreuse branch March 16, 2017 19:42

petrochenkov mentioned this pull request Jan 31, 2020
rustbuild: LLVM is not partially rebuilt anymore #68715
Closed

Uh oh!

Uh oh!

Conversation

petrochenkov commented Mar 7, 2017

Uh oh!

alexcrichton commented Mar 7, 2017

Uh oh!

alexcrichton commented Mar 7, 2017

Uh oh!

petrochenkov commented Mar 8, 2017

Uh oh!

alexcrichton commented Mar 8, 2017

Uh oh!

bors commented Mar 8, 2017

Uh oh!

bors commented Mar 9, 2017

Uh oh!

alexcrichton commented Mar 9, 2017

Uh oh!

alexcrichton commented Mar 10, 2017

Uh oh!

bors commented Mar 10, 2017

Uh oh!

alexcrichton commented Mar 10, 2017

Uh oh!

bors commented Mar 10, 2017

Uh oh!

bors commented Mar 11, 2017

Uh oh!

petrochenkov commented Mar 11, 2017

Uh oh!

bors commented Mar 11, 2017

Uh oh!

bors commented Mar 12, 2017

Uh oh!

bors commented Mar 12, 2017

Uh oh!

petrochenkov commented Mar 12, 2017

Uh oh!

bors commented Mar 12, 2017

Uh oh!

bors commented Mar 12, 2017

Uh oh!

bors commented Mar 12, 2017

Uh oh!

petrochenkov commented Mar 12, 2017

Uh oh!

bors commented Mar 12, 2017

Uh oh!

bors commented Mar 12, 2017

Uh oh!

petrochenkov commented Mar 12, 2017

Uh oh!

bors commented Mar 12, 2017

Uh oh!

bors commented Mar 12, 2017

Uh oh!

petrochenkov commented Mar 13, 2017

Uh oh!

bors commented Mar 13, 2017

Uh oh!

bors commented Mar 13, 2017

Uh oh!

alexcrichton commented Mar 13, 2017

Uh oh!

bors commented Mar 13, 2017

Uh oh!

bors commented Mar 13, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants