Usage of pthread in a Node.js app never lets it exit

[RReverser]

There is something wrong with the way Emscripten uses threads on Node.js. Looks like any usage prevents Node app from ever exiting.

Just tried the simplest example:

#include <thread>
#include <iostream>

int main() {
	std::thread t([] {
		std::cout << "Hello from another thread\n";
	});
	t.join();
	return 0;
}

Compiled with:

> emcc temp.cc -o temp.js -pthread -s PTHREAD_POOL_SIZE=4

This results in:

> node --experimental-wasm-threads --experimental-wasm-bulk-memory temp
Hello from another thread
Pthread 0x705d10 exited.
[stuck here]

Looking at https://nodejs.org/api/worker_threads.html, I suspect Emscripten needs to call .unref() to indicate that it's okay to exit as soon as reference is unreachable, but probably doesn't do that yet?

[emaxx-google]

Is this supposed to work without PROXY_TO_PTHREAD? See https://emscripten.org/docs/porting/pthreads.html#additional-flags

[RReverser]

@emaxx-google Yes. PROXY_TO_PTHREAD is a separate feature that is built on top of PThread emulation and allows to move main to a separate thread. This bug is about regular PThread emulation.

[emaxx-google]

I'm not an Emscripten expert, but the same page describes in a bit more detail:

The Emscripten implementation for the pthreads API should follow the POSIX standard closely, but some behavioral differences do exist:

When pthread_create() is called, if we need to create a new Web Worker, then that requires returning the main event loop. That is, you cannot call pthread_create and then keep running code synchronously that expects the worker to start running - it will only run after you return to the event loop. This is a violation of POSIX behavior and will break common code which creates a thread and immediately joins it or otherwise synchronously waits to observe an effect such as a memory write.

[RReverser]

@emaxx-google That's not the issue here. I already know how pthreads work in Emscripten :) What that paragraph is describing is the reason why you need PTHREAD_POOL_SIZE which I'm already passing.

[juj]

Try building with -s PTHREAD_POOL_SIZE=1 -s EXIT_RUNTIME=1 linker flags. Does that help?

[RReverser]

@juj Hmm it does, but that's not necessary in a non-pthread builds. Why is adding pthread different?

[RReverser]

[semi-answering my own question] IIRC the difference is that EXIT_RUNTIME forcibly does process.exit() or something like that, which is not exactly what we want here - what we really want is for Node to run event loop to completion and exit naturally, but right now it seems it can't due to hanging workers.

[juj]

The behavior is likely caused by this line:

emscripten/src/postamble.js

Line 441 in 5568b81

noExitRuntime = true;

. If you remove that line, it should also cause the code to exit, even if you don't specify -s EXIT_RUNTIME=1. Does that also do a forcible process.exit()?

You can also try running PThread.terminateAllThreads(); JS function if you think the issue is caused by hanging Workers. That will tear down all Workers.

[RReverser]

. If you remove that line, it should also cause the code to exit, even if you don't specify -s EXIT_RUNTIME=1. Does that also do a forcible process.exit()?

Yeah, they're equivalent AFAIK:

emscripten/src/postamble.js

Line 197 in c2462cd

exit(ret, /* implicit = */ true);

[juj]

That should be the only difference with respect to process teardown in st and mt builds. If in that mode there is a forcible shutdown with process.exit() happening, then it should also be happening in non-multithreaded builds. Not sure where such a forcible shutdown is happening, though Emscripten at least does not emit a call to process.exit() API into the generated build.

[RReverser]

I actually now wonder if this is "expected behaviour"... if issue is caused by hanging Workers, then there are only two choices:

1. Emscripten tears them down at the end of main. Node.js exits gracefully when JS is used as a process, but if user imports this JS as a module and tries to call other functions, they can no longer [easily] create threads because Worker pool is empty.
2. Emscripten doesn't tear them down, Workers remain hanging and can be used by further calls into the module, but when used as a process, Node.js can't shut down - this is what happens now.

I do think that it's possible to get a middle ground by looking into those .unref() - Node.js should be smart enough to figure out that, when main JS has finished execution, workers can no longer receive messages and don't need to wait anymore. Right now it seems like they hang around in case something will send them a message and they need to do more work.

[RReverser]

So yeah, I can reproduce by only loading the .worker.js part from a custom JS instead of Emscripten glue:

const {Worker} = require('worker_threads');
let w = new Worker('./temp.worker.js');

This also hangs forever, probably because Worker still waits for messages.

[RReverser]

@addaleax I admit I don't fully understand the .ref() / .unref() mechanics of worker_threads, maybe you can help?

Basically, the question is - is there a way to make sure that Workers are torn down once the Worker object is GC'd?

Right now it looks like a reference cycle keeping both main thread and the Worker alive.

[addaleax]

Basically, the question is - is there a way to make sure that Workers are torn down once the Worker object is GC'd?

No, that never happens. Worker objects cannot be GC’ed before they are terminated because they are GC roots because they receive events from the event loop.

I admit I don't fully understand the .ref() / .unref() mechanics of worker_threads, maybe you can help?

If you call .unref() on a Worker, then that means that they will not keep the event loop alive on its own, i.e. this works the same as .ref()/unref() on network sockets, timers, etc.

This also hangs forever, probably because Worker still waits for messages.

That sounds very likely, yes, but I wouldn’t know what we could do about this on the Node.js side.