build: enable xz compressed tarballs where possible (v0.12 and v0.10)

[rvagg]

Continuing on from nodejs/build#284

/cc @chorrell, @mscdex, @ljharb, @nodejs/build

We didn't actually agree on anything there but I'm in a why not mood right now.

This will mean that we get .xz tarballs from v0.12.10 and v0.10.42 onwards for those lines. Backing up and providing .xz tarballs for the others is impossible unless we exclude them from the signed SHASUMS file, which is far from ideal so I'm a strong -1 on completing v0.12 and v0.10. Tools like nvm will just have to hardwire in versions numbers for determining when to use .xz and when to use .gz. We're going to have to do something similar in node-gyp to use header files for older versions now that we have them fixed (the same versions will finally have a proper header tarball that can be used).

[rvagg]

this logic is the same as master FYI

$ git diff -U0 master -- Makefile| grep -i xz | wc -l
0

[bnoordhuis]

LGTM FWIW. I'd be okay with making it non-conditional, mandating that xz is installed is not an onerous requirement.

[rvagg]

@bnoordhuis as per the notes, it's annoying on OSX, hence the conditional

[bnoordhuis]

I saw that but I assume the OS X buildbots will have it installed so when would it be an issue? People don't generally run it manually, do they?

[rvagg]

Yeah, mainly just the release build machines and this is one of the criteria for those, I don't think it gets run much in the wild but it can be used to build tarballs if you download source and I suspect there are come companies out there doing that, unlikely that they'd be doing it for OSX, however!

[jbergstroem]

LGTM here too. I don't mind the conditional much.

[richardlau]

I'd prefer the conditional to be consistent with the current v4.x.

In addition to OS X, AIX also does not have xz installed by default.

[chorrell]

Great!

[jasnell]

LGTM

[ljharb]

so in semver-speak ~0.12.10 || ~0.10.42 || >= 1.0 covers the node versions that definitely will have xz archives available?

[jbergstroem]

@ljharb I think it should be ~0.12.10 || ~0.10.42 || >= 2.3.2.

[ljharb]

@rvagg any possibility of completing >= 1.0.0 with xz?

[jbergstroem]

@ljharb that'd be painful. Is it important? I mean, 0.10, 0.12 starts midway too.

[ljharb]

I don't have visibility into how painful it is for you folks - but every version i have to hardcode into nvm is painful in a long-term sense, whereas creating the xz's seems like it would be painful once.

[jbergstroem]

@ljharb last I checked we did that xz/version check where we just hardcode lowest required versions. Shouldn't that be an ok compromise?

[ljharb]

Yes, currently we hardcode 1 version. The current suggestion seems to indicate I'd have to hardcode 3. Just trying to minimize how many I have to manage.

[rvagg]

OK, this is a little bit complicated and there are multiple concerns here, both for version managers and for node-gyp. I believe the correct ranges are:

• Linux (all) and sources with .xz compression: ~0.12.10 || ~0.10.42 || >= 1.0
• Linux (all), sources and darwin with .xz compression: ~0.12.10 || ~0.10.42 || >= 2.3.2
• Valid headers tarball (with .xz): ~0.12.10 || ~0.10.42 || >= 2.3.2

i.e.

• https://iojs.org/download/release/v1.0.0/ up to https://iojs.org/download/release/v2.3.1/ has .xz except for darwin and no headers
• https://iojs.org/download/release/v2.3.2/ adds .xz for darwin and a usable headers tarball
• https://nodejs.org/download/release/v0.12.7/ and https://nodejs.org/download/release/v0.10.40/ and prior are classic, no extra goodies
• https://nodejs.org/download/release/v0.10.41/, https://nodejs.org/download/release/v0.12.8/ and https://nodejs.org/download/release/v0.12.9/ have flawed header files, they are too big and aren't restricted to just headers, so node-gyp won't be using these
• https://nodejs.org/download/release/v0.10.42/ and https://nodejs.org/download/release/v0.12.9/ don't yet exist but will within a week and should have .xz and usable header files

Clear as mud?

[ljharb]

Thank you for documenting it. Any "backfills" you can do that will condense those three rows, or their resulting columns, into a smaller number of combinations than 9, would be appreciated :-)

[rvagg]

• landed in v0.12-staging @ a305339
• landed in v0.10-staging @ b125512
• v0.12 nightly building @ https://ci.nodejs.org/job/iojs+release/378/
• v0.10 nightly building @ https://ci.nodejs.org/job/iojs+release/379/

[rvagg]

• https://nodejs.org/download/nightly/v0.10.42-nightly20160128b125512a5c/
• https://nodejs.org/download/nightly/v0.12.10-nightly20160128a305339f66/

Every tarball has an xz and the headers are 🅰️1️⃣