build: Updates for AIX npm support - part 1

[mhdawson]

This PR is the first step enabling support for native modules
for AIX. The main issue is that unlike linux where all
symbols within the Node executable are available to the shared
library for a native module (npm), on AIX the symbols must
be explicitly exported. In addition, when the shared library is
built it must be linked using a list of the available symbols.

This patch covers the changes need to:

1. Export the symbols when building the node executable
2. Generate the file listing the symbols that can be used when
   building the shared library.

For AIX, it breaks the build process into 2 steps. The first builds
a static library and then generates a node.exp file which contains
the symbols from that library. The second builds the node executable
and uses the node.exp file to specify which symbols should be
exported. In addition, it save the node.exp file so that it can
later be used in the creation of the shared library when building
a native module.

The following additional steps will be required in dependent projects
to fully enable AIX for native modules and are being worked
separately:

• Updates to node-gyp to use node.exp when creating the
  shared library for a native module
• Fixes to gyp related to copying files as covered in
  https://codereview.chromium.org/1368133002/patch/1/10001
• Pulling in updated gyp versions to Node and node-gyp
• Pulling in latest libuv

These changes were done to minimize the change to other platforms
by working within the existing structure to add the 2 step process
for AIX without changing the process for other platforms.

[rvagg]

could we clean this up a little, some missing spaces and it's quite long, maybe something like:

 find $1 -name "*.a" \
| xargs nm -Xany -BCpg \
| awk '{
if ((($2 == "T") || ($2 == "D") || ($2 == "B")) && (substr($3,1,1) != ".")) { print $3 }
}' \
| sort -u >> $2

Also perhaps a comment at the top of the file that this if for creating an AIX .exp file, just to give some contents to the next poor dev who goes fishing around in tools/ and stumbles on our crazy assortment of hacks

[mhdawson]

sure, once we settle any other issues I'll fix this up/

[bnoordhuis]

Would it make sense to split the current 'node' target in two: one that builds libnode.a, one that builds the binary? AIX-specific post-processing could then be done with less duplication.

[Fishrock123]

Curious: does npm even work correctly on AIX?

[S-YOU]

Not sure what kind of python it is, but this should be if flavor in ('aix',):?

[thefourtheye]

Very true. I would say change it to if flavor == 'aix'

[mhdawson]

I'd followed the style in the file just a few lines down, but given that we only expect one in the set the == 'aix' is probably better, which change

[S-YOU]

👍

[mhdawson]

@Fishrock123 in response to the "does npm work for AIX" the answer is yes given the full set of changes we have in the IBM releases which is what I'm starting to contribute. There are some modules that use binaries which are platform specific but otherwise native modules can be installed/compiled.

[mhdawson]

@bnoordhuis the idea of modifying to split for all platforms makes sense to me but I did not want to presume we'd do this to accommodate AIX.

[bnoordhuis]

It seems cleaner than the current approach so I don't think anyone will stop you.

[mhdawson]

Thanks for all the good feedback, sorry I'm not had time to update yet but I hope to get to it in the next few days.

[mhdawson]

Just testing updates to address most of the comments will post tomorrow if there are no issues.

I've looked a bit at changing so all platforms build the lib and then the executable but I have a few concerns:

1. not sure how to keep the name of the static library the same as before as I think it is based on the target name (may just be my lack of familiarity with gyp)
2. there seem to be some platform specific options/configuration in the node target. I don't think we have tests that the static build case so I'm a little worried that the changes might break those platforms with a 2 step process and we don't have a good way to test. Similar issues might exist for AIX but since support for AIX is new it would not be a regression if the static build does not work.

@bnoordhuis any thoughts ?

[bnoordhuis]

there seem to be some platform specific options/configuration in the node target. I don't think we have tests that the static build case so I'm a little worried that the changes might break those platforms with a 2 step process and we don't have a good way to test.

The CI would catch that, wouldn't it? Maybe I misunderstand what you're worried about.

That said, while I think the two phase approach would be a little neater due to less duplication, it's not like your current approach is abjectly terrible. Go with whatever you're comfortable with.

[mhdawson]

Ok, at this point I think I'd like to stick with what we have instead of changing the other platforms. We want to make progress on nodejs/roadmap#9 so I think we can revisit as part of that.

I've pushed an update to address the comments and the tests pass my local CI run, @rvagg, @bnoordhuis, @S-YOU, @mscdex, @thefourtheye can you take another look.

[S-YOU]

sorry again, but may be startswith like this is better than in?

[mhdawson]

Makes sense to me, updated

[S-YOU]

👍

[mhdawson]

@S-YOU updated to reflect your comment.

[bnoordhuis]

Style nit: can you break up this line?

[bnoordhuis]

LGTM with nit.

[mhdawson]

Updated to address Ben's comment and include one small change we found was needed as part of our internal testing. Given Ben's lgtm I'll plan to pull in, in the next day or so

[mhdawson]

CI run in advance of landing - https://ci.nodejs.org/job/node-test-pull-request/579/

[mhdawson]

CI does not look to have any related failures

Squashed down to 1 commit

[mhdawson]

Landed as 15bcbf8