Crashes on freebsd 8

[p]

Running the following script:

reactor% cat server.coffee
express = require 'express'

repeatedly:

reactor% coffee server.coffee


#
# Fatal error in ../deps/v8/src/store-buffer.cc, line 82
# CHECK(old_virtual_memory_->Commit( reinterpret_cast(old_start_), (old_limit_ - old_start_) * kPointerSize, false)) failed
#


==== Stack trace is not available ==========================


==== Isolate for the thread is not initialized =============

zsh: abort (core dumped) coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
FATAL ERROR: v8::Context::New() V8 is no longer usable
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
zsh: segmentation fault (core dumped) coffee server.coffee
reactor% coffee server.coffee
FATAL ERROR: v8::Context::New() V8 is no longer usable
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee
reactor% coffee server.coffee


#
# Fatal error in ../deps/v8/src/store-buffer.cc, line 98
# CHECK(virtual_memory_->Commit(reinterpret_cast(start_), kStoreBufferSize, false)) failed
#


==== Stack trace is not available ==========================


==== Isolate for the thread is not initialized =============

zsh: abort (core dumped) coffee server.coffee

This also happens with plain js, npm, etc.

The crash appears to happen very early during startup process, before any userland code is executed.

It seems that when the runtime does not crash during initialization, it works fine. E.g. I am able to install packages, express does work.

reactor% node -v
v0.8.15
reactor% uname -a
FreeBSD reactor.universe 8.2-STABLE FreeBSD 8.2-STABLE #0: Sun Sep 11 14:34:49 EDT 2011 build@reactor.universe:/home/build/obj/nfs/src/sys/REACTOR i386

[bnoordhuis]

Can you try master? I've landed several FreeBSD and DragonFlyBSD fixes in the last months.

FWIW, I run the amd64 9.1 release candidate and I get a 100% score on make test.

[p]

Still crashes.

reactor% while true; do node empty.js; done
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
#
# Fatal error in ../deps/v8/src/store-buffer.cc, line 98
# CHECK(virtual_memory_->Commit(reinterpret_cast(start_), kStoreBufferSize, false)) failed
#
==== Stack trace is not available ==========================
==== Isolate for the thread is not initialized =============
#
# Fatal error in ../deps/v8/src/store-buffer.cc, line 98
# CHECK(virtual_memory_->Commit(reinterpret_cast(start_), kStoreBufferSize, false)) failed
#
==== Stack trace is not available ==========================
==== Isolate for the thread is not initialized =============
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
# CHECK(old_virtual_memory_->Commit( reinterpret_cast(old_start_), (old_limit_ - old_start_) * kPointerSize, false)) failed
#
==== Stack trace is not available ==========================
==== Isolate for the thread is not initialized =============
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
#
# Fatal error in ../deps/v8/src/store-buffer.cc, line 82
# CHECK(old_virtual_memory_->Commit( reinterpret_cast(old_start_), (old_limit_ - old_start_) * kPointerSize, false)) failed
#
==== Stack trace is not available ==========================
==== Isolate for the thread is not initialized =============
#
# Fatal error in ../deps/v8/src/store-buffer.cc, line 98
# CHECK(virtual_memory_->Commit(reinterpret_cast(start_), kStoreBufferSize, false)) failed
#
==== Stack trace is not available ==========================
==== Isolate for the thread is not initialized =============
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
FATAL ERROR: v8::Context::New() V8 is no longer usable
^CFATAL ERROR: v8::Context::New() V8 is no longer usable
^Zzsh: suspended while true; do; node empty.js; done
reactor% kill %%
reactor% 
[1] + terminated while true; do; node empty.js; done
reactor% node -v
v0.9.4-pre
reactor% cat empty.js 

[p]

At some point I received a message about running out of memory. I will note precise wording if and when it appears again.

[p]

Make test results: http://pastebin.com/7KKzcXvd

Most of the time make test hangs and never finishes. It hangs at different points on different runs. One time make test ran to the end: http://pastebin.com/2Uf08mas

[p]

FATAL ERROR: Evacuation Allocation failed - process out of memory

[p]

FATAL ERROR: Evacuation Allocation failed - process out of memory

This happens not just during initialization. My kue ui process just died with that. (https://github.com/learnboost/kue)

[bnoordhuis]

How much virtual memory is available? The error you're seeing means V8 tries to mmap() memory and fails. It's likely a problem with your setup, not with V8 or node.js.

[p]

Currently:

Mem: 1828M Active, 760M Inact, 282M Wired, 29M Cache, 112M Buf, 65M Free
Swap: 3072M Total, 184K Used, 3072M Free

[bnoordhuis]

How much virtual memory is available on a per-process basis? (That's ulimit -v.)

Do you have the same issue when you clone and build https://github.com/v8/v8.git? Build it with gmake <arch> and run the test suite with python tools/run-tests.py --arch-and-mode=<arch> mjsunit, where arch is ia32.release or x64.release.

[p]

reactor% ulimit -v
unlimited

Will get back to you on v8.

[p]

Couldn't build v8.

First I had http://pastebin.com/kSduHuG0. Then I removed -Werror and it went until http://pastebin.com/T8UH5WFn. This I could not immediately fix.

reactor% gcc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix
gcc version 4.2.2 20070831 prerelease [FreeBSD]

[bnoordhuis]

First I had http://pastebin.com/kSduHuG0. Then I removed -Werror and it went until http://pastebin.com/T8UH5WFn. This I could not immediately fix.

I've sent two patches upstream: https://codereview.chromium.org/11595004 and https://codereview.chromium.org/11590005

The branch is here: https://github.com/bnoordhuis/v8/compare/freebsd-fixups

[bnoordhuis]

Target: i386-undermydesk-freebsd

You should try upgrading to 64 bits. There's an issue with 32 bits builds on FreeBSD in that there's relatively little address space available to V8 (less so than on, say, 32 bits Linux); see 4165f73 for more details.

As you can see, we've landed a fix but it's up for debate how effective it is. The issue simply doesn't exist with 64 bits build because the address space is so mind bogglingly big.

[p]

The 64 bit upgrade has been pending for a while as the machine has 8 gb of memory in it. I started rebuilding another machine into 9.1/i386 as that is cheaper in the short term.

If this is a known issue on 32-bit freebsd, shouldn't it be advertised somewhere? Certainly it would have saved me a lot of frustration.

Issue or not, the process should not crash if it cannot allocate memory but exit with at least a meaningful error message?

Also, when it does exit as opposed to segv or abort, it exits with code 1 which makes it really difficult to distinguish crashes from legitimate exits.

[p]

Thinking about this issue again, how exactly does one run out of 2 gb of address space while executing an empty file?