test: reduce memory use in test stringbytes

[mcornac]

When I run the string bytes tests dealing with strings of length kStringMaxLength or more on a Windows test machine with 4GB memory I see the tests skipped or failed with the following outputs respectively.

• 1..0 # Skipped: intensive toString tests due to memory confinements
• RangeError: Invalid array buffer length

The code being used to determine if the test should be skipped is:

 try {
new Buffer(kStringMaxLength * 3);
} catch(e) {
assert.equal(e.message, 'Invalid array buffer length');
console.log(
'1..0 # Skipped: intensive toString tests due to memory confinements');
return;
}

A problem with this approach is that the garbage collector does not necessarily run between the tried allocation and the necessary allocations later. That means that even though no exception is thrown in the try block since the machine has enough memory, the later allocations fail since the memory is now taken.

To verify this, I ran the test with Release\node.exe --expose-gc test\parallel\test-stringbytes-external-at-max.js to expose global.gc() which I used after the try catch block. With this setup the test would either skip or succeed as desired.

To solve the issue, I removed the Buffer allocations that were used to check if the tests should be skipped and moved the actual Buffer allocations into the try block so the test can still be skipped if memory allocation fails here. The tests should now use less total memory and be less likely to be skipped and more importantly, less likely to fail due to a memory constraint when they are not skipped.

[targos]

The fix looks good, but why did you change the assertions in the catch blocks?

[mcornac]

I'm not sure if there are disadvantages but I thought it could be helpful to rethrow the original exception in case a different exception occurs.
E.g.

/sandbox/cornacch/node/test/parallel/test-stringbytes-external-at-max.js:15
if (e.message != 'Invalid array buffer length') throw (e);
^
Error: toString failed
at Buffer.toString (buffer.js:382:11)
at Object.<anonymous> (/sandbox/cornacch/node/test/parallel/test-stringbytes-external-at-max.js:13:17)

Instead of

assert.js:89
throw new assert.AssertionError({
^
AssertionError: 'toString failed' == 'Invalid array buffer length'
at Object.<anonymous> (/sandbox/cornacch/node/test/parallel/test-stringbytes-external-at-max.js:15:10)

[mhdawson]

lgtm, I do think re-throwing makes sense so that we can see the original exception

[trevnorris]

The reason behind the current implementation is that it helps account for the additional space in the v8 heap necessary to store all the strings that are created. I'd say the most reliable approach would be to run gc right after the try/catch.

[mcornac]

@trevnorris I made some changes to the at-max test to use --expose-gc based on some other tests. Can you please take a look before I update the other tests?

[trevnorris]

@thinkingdust Nice. Like the way you did that.

[mcornac]

Thanks. I've pushed the changes to the other files. I also ran the tests on the Windows machine again and they always either skip or pass now as desired.

[trevnorris]

I'd squash the commits and include a note about why the change to throwing instead of assert.

Other than that, LGTM.

[bnoordhuis]

I'd add a comment explaining you don't use assert in order to preserve the original exception.

[bnoordhuis]

Oh, and a femto style nit: !==

[bnoordhuis]

Some style nits but basic premise LGTM.

[mcornac]

@bnoordhuis Thanks for the suggestions, updated.

[bnoordhuis]

LGTM. CI: https://ci.nodejs.org/job/node-test-pull-request/693/

[jasnell]

Kicked off a new CI run... the previous run (693) was returning a 404: https://ci.nodejs.org/job/node-test-pull-request/712/

[targos]

LGTM

[mcornac]

I think there is only one related failure.
test-stringbytes-external-exceed-max-by-1-base64.js failed with a timeout here:
https://ci.nodejs.org/job/node-test-binary-arm/469/RUN_SUBSET=1,nodes=pi1-raspbian-wheezy/console
but the same test passed here:
https://ci.nodejs.org/job/node-test-binary-arm/469/RUN_SUBSET=1,nodes=pi2-raspbian-wheezy/console

[Fishrock123]

Maybe just exclude this test on low memory machines? Something like

if(os.totalmem()<=512000000/* 512mb */){// bailreturn;}

(note: totalmem() returns bytes)

Edit actually maybe freemem() would be better? Idk.

[mhdawson]

So if I understand correctly @thinkingdust we still saw a failure in the test even after the PR ?

[mcornac]

It's a 404 now but there was a timeout failure in one of the tests on Raspberry Pi.
I don't know how v8 decides if a Buffer allocation is valid. The test needs about 756MB (3 * kStringMaxLength) while I'm guessing the Pi has 512MB but it seems like the test was not skipped but instead ran very slowly.

[trevnorris]

@thinkingdust v8 doesn't have anything to do with memory allocation with Buffers. It's a simple malloc() call, and v8 uses whatever the system returns. In these cases I believe that the allocation is coming from swap space, hence why operations on the memory returned is incredibly slow. I think @Fishrock123's idea is worth trying out.

[mcornac]

I added a freemem check for the at-max test and it works fine on my Linux machine. I can try it on the Windows machine tomorrow. Is this use of freemem reliable enough to remove the try catch blocks?

[mcornac]

Using the freemem check works to always skip or pass on the original Windows machine.

[mcornac]

@trevnorris I changed the at-max test to check using freemem instead of try/catch and gc. If that looks okay to you I'll update the other files too.

[mhdawson]

Another failure on ARM today: https://ci.nodejs.org/job/node-test-binary-arm/81/RUN_SUBSET=4,nodes=pi2-raspbian-wheezy/console

Would be nice to get the fix in.

[jasnell]

@trevnorris ... can you take another look at this when you get a moment? :-)

[Fishrock123]

Some things: I think freemem may include cache and possibly swap, but I can confirm totalmem is just the hardware ram.

I think my original suggestion of just skipping on 512mb ram machines (ontop of the other protections) would be best.

[trevnorris]

Had a follow up to @Fishrock123's comment, but still LGTM.

[mcornac]

@trevnorris@Fishrock123 I added common.kMaxTestMemUsage. Can you take a look?

[jasnell]

LGTM

[Fishrock123]

Correct me if I'm wrong but I think kMaxTestMemUsage needs to be set-able from C++ to be of value?

[Fishrock123]

Or at least an environment variable.

[trevnorris]

@Fishrock123 Don't think so. We've honestly just selected the value based on past fail cases. Can't see why it would need to be configurable and/or need to be set from C++.

[Fishrock123]

Ah, ok. I think exports.enoughTestMem = os.totalmem() > 0x20000000; might be better, but maybe don't hold the PR on it.

[trevnorris]

So make the export simply a boolean? Can't say I'm against that, but also don't want to make @mhdawson @thinkingdust rage quit this PR. ;-)

EDIT: auto-select fail

[mcornac]

@Fishrock123@trevnorris

[Fishrock123]

Rubber-stamp LGTM

[mcornac]

🎆 ✨

[trevnorris]

LGTM

[trevnorris]

@thinkingdust i'm having a hard time segmenting these commits into discrete changes. Mind if I squash them all, or would you like to take care of it?

[mcornac]

I think 1 commit works. Squashed.

[trevnorris]

Thanks much! Landed in a40b9ca