Skip to content

Remove /MP from default additonal options - #22661

Closed
skelliam wants to merge 3 commits into
nodejs:masterfrom
skelliam:patch-2
Closed

Remove /MP from default additonal options#22661
skelliam wants to merge 3 commits into
nodejs:masterfrom
skelliam:patch-2

Conversation

@skelliam

Copy link
Copy Markdown
Contributor

Forcing this option means that the user-provided option MultiProcessorCompilation in msvs_settings (from, for example, binding.gyp) is useless. It also means that one cannot use #import in their source code or they will be faced with this error when trying to build:

error C2813: #import is not supported with /MP (compiling source file...)

Please see additional discussion of this here: nodejs/node-gyp#1087
and here: nodejs/node-gyp#26

@nodejs-github-botnodejs-github-bot added the build Issues and PRs related to build files or the CI. label Sep 2, 2018
@addaleax

Copy link
Copy Markdown
Member

@nodejs/platform-windows @nodejs/node-gyp

@refackrefack added the windows Issues and PRs related to the Windows platform. label Sep 2, 2018
@refack

Copy link
Copy Markdown
Contributor

/CC @nodejs/build-files

This makes sense to me. It's a low value optimization, and if it has negative effects (re #import) we can remove this.

refack
refack previously approved these changes Sep 2, 2018

@refackrefack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still LGTM

@skelliam

Copy link
Copy Markdown
ContributorAuthor

There is one more instance; can I add on to this commit?

@tniessen

Copy link
Copy Markdown
Member

Will this make building Node.js itself even slower on Windows?

@skelliam

Copy link
Copy Markdown
ContributorAuthor

@tniessen , if you look at the comment in nodejs/node-gyp#1087, it appears that Ninja already handles parallelization, and in fact it looks like there is some effort filter /MP out anyway.

@tniessen

Copy link
Copy Markdown
Member

cc @seishun who worked on this flag before.

@skelliam

Copy link
Copy Markdown
ContributorAuthor

Here is the documentation from MSDN regarding #import with /MP.

@refack

Copy link
Copy Markdown
Contributor

FYI: GYP allows for removing items from list with the ! suffix, so the following msvs_settings block removes the /MP:

{
'targets': [
{
'target_name': 'binding',
'defines': [ 'V8_DEPRECATION_WARNINGS=1' ],
'sources': [ 'binding.cc' ],
'msvs_settings': {
'VCCLCompilerTool': {
'AdditionalOptions!': [
'/MP'
]
}
}
}
]
}

@refack

refack commented Sep 2, 2018

Copy link
Copy Markdown
Contributor

Will this make building Node.js itself even slower on Windows?

Highly unlikely, since even when you use MSBuild it does multiprocess compilation by default, and we ask it to use at least two in vcbuild.bat:

node/vcbuild.bat

Lines 300 to 305 in 9f7efd5

set"msbcpu=/m:2"
if"%NUMBER_OF_PROCESSORS%"=="1"set"msbcpu=/m:1"
set"msbplatform=Win32"
if"%target_arch%"=="x64"set"msbplatform=x64"
if"%target%"=="Build"ifdefined no_cctest settarget=node
msbuild node.sln %msbcpu% /t:%target% /p:Configuration=%config% /p:Platform=%msbplatform% /clp:NoSummary;NoItemAndPropertyList;Verbosity=minimal /nologo

P.S. /MP is probably incompatible with clcache

@skelliam

Copy link
Copy Markdown
ContributorAuthor

Thanks @refack; was looking all over for a solution to this, sounds like your proposal would have worked. I did try to set the contents of AdditionalOptions to an empty string, but as mentioned in the linked discussion, this is an additive option.

@refack

refack commented Sep 2, 2018

Copy link
Copy Markdown
Contributor

AdditionalOptions to an empty string, but as mentioned in the linked discussion, this is an additive option.

BTW for that GYP has the = suffix, so:

'msvs_settings': { 'VCCLCompilerTool': { 'AdditionalOptions=': []
} }

@targos

Copy link
Copy Markdown
Member

@tniessen

Will this make building Node.js itself even slower on Windows?

Building Node.js on Windows is not as slow as it used to be. There were recent improvements.

@bzoz

bzoz commented Sep 3, 2018

Copy link
Copy Markdown
Contributor

I'm -1. On my box this increases clean build time from 9 min 22 sec to 14 min 21 sec.

seishun
seishun previously requested changes Sep 3, 2018

@seishunseishun left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-1 if it slows down compilation as @bzoz described.

@richardlau

Copy link
Copy Markdown
Member

Longer term the better solution is probably to implement nodejs/node-gyp#1118 and stop node-gyp using core's common.gypi.

@joaocgreisjoaocgreis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make compiling core much slower. I see CI hasn't yet been run for this PR, but it should be very noticeable there. If there's a way to make /MP active only for node core then this might be worth considering.

@skelliam

Copy link
Copy Markdown
ContributorAuthor

Can't you still have /MP with the XML option "MultiProcessorCompilation" in msvs_settings? This way the user override is at least more intuitive (IMO).

Otherwise I like the idea from #1118.

@refack
refack dismissed their stale reviewSeptember 3, 2018 21:27

Not until we find how to offset

@refack

Copy link
Copy Markdown
Contributor

I see CI hasn't yet been run for this PR, but it should be very noticeable there.

I did:
Pre on test-rackspace-win2012r2-x64-12 — 35m
Post on test-rackspace-win2012r2-x64-11 — 50m
34% increase in time. But if this is an issue of higher parallelism, I have a feeling we could offset that with a higher number in

set"msbcpu=/m:2"

@refack

Copy link
Copy Markdown
Contributor

Can't you still have /MP with the XML option "MultiProcessorCompilation" in msvs_settings? This way the user override is at least more intuitive (IMO).

I think that makes sense.
👍

this will replace the hardcoded /MP with an implicit /MP that is easier for users to override.
@skelliam

Copy link
Copy Markdown
ContributorAuthor

Added 'MultiProcessorCompilation': 'true' (implicit /MP) instead of the hardcoded /MP. Users can override this with 'MultiProcessorCompilation': 'false' in their own binding.gyp.

@refackrefack left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯 Since this is essentially a no-op.

@richardlaurichardlau left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the change from /MP to 'MultiProcessorCompilation': 'true' provided build times remain unchanged.

@BridgeARBridgeAR added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Sep 5, 2018
@BridgeAR

Copy link
Copy Markdown
Member

bzoz
bzoz approved these changes Sep 5, 2018
@tniessen

Copy link
Copy Markdown
Member

The compilation time on Windows did not increase after the latest CI run.

@tniessen

Copy link
Copy Markdown
Member

Landed in 56d9cd4.

@tniessentniessen closed this Sep 9, 2018
tniessen pushed a commit that referenced this pull request Sep 9, 2018
PR-URL: #22661
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
targos pushed a commit that referenced this pull request Sep 10, 2018
PR-URL: #22661
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
@skelliam
skelliam deleted the patch-2 branch September 10, 2018 18:54
@targostargos mentioned this pull request Sep 18, 2018
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author readyPRs that have at least one approval, no pending requests for changes, and a CI started.buildIssues and PRs related to build files or the CI.windowsIssues and PRs related to the Windows platform.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

11 participants

@skelliam@addaleax@refack@tniessen@targos@bzoz@richardlau@BridgeAR@joaocgreis@seishun@nodejs-github-bot