std.os.ChildProcess should use clone instead of fork on linux

[andrewrk]

vfork pauses the parent process until the child process either calls execve or exit. This prevents Copy On Write pages, which makes spawning child processes avoid double the parent resident memory requirements when overcommit is off.

[andrewrk]

Interesting notes from the man page:

NOTES
 Some consider the semantics of vfork() to be an architectural blemish,
 and the 4.2BSD man page stated: "This system call will be eliminated
 when proper system sharing mechanisms are implemented. Users should
 not depend on the memory sharing semantics of vfork() as it will, in
 that case, be made synonymous to fork(2)." However, even though modern
 memory management hardware has decreased the performance difference
 between fork(2) and vfork(), there are various reasons why Linux and
 other systems have retained vfork():

 * Some performance-critical applications require the small performance
 advantage conferred by vfork().

 * vfork() can be implemented on systems that lack a memory-management
 unit (MMU), but fork(2) can't be implemented on such systems.
 (POSIX.1-2008 removed vfork() from the standard; the POSIX rationale
 for the posix_spawn(3) function notes that that function, which pro‐
 vides functionality equivalent to fork(2)+exec(3), is designed to be
 implementable on systems that lack an MMU.)

 * On systems where memory is constrained, vfork() avoids the need to
 temporarily commit memory (see the description of /proc/sys/vm/over‐
 commit_memory in proc(5)) in order to execute a new program. (This
 can be especially beneficial where a large parent process wishes to
 execute a small helper program in a child process.) By contrast,
 using fork(2) in this scenario requires either committing an amount
 of memory equal to the size of the parent process (if strict over‐
 committing is in force) or overcommitting memory with the risk that
 a process is terminated by the out-of-memory (OOM) killer.

[bnoordhuis]

I don't think that can work. With vfork() you're only allowed to call execve() or _exit(). The setup that spawnPosix() does, like changing uid/gid and the cwd, is incompatible with vfork() - it would be visible to other threads in the parent process.

[andrewrk]

My understanding is that changes to memory would affect the parent process, but the actual pid is a new child pid. So things like current directory, uid/gid, would be independent from the parent process. I'm doing some experiments now. Currently I'm getting a confusing segfault with the following code which I am troubleshooting:

const std = @import("std");
const os = std.os;
const linux = os.linux;

pub fn main() void {
 var x: i32 = 1234;
 const rc = linux.vfork();
 if (rc == 0) {
 // we are the child
 x += 1;
 linux.exit(0);
 }
 std.debug.warn("x={}\n", x);
}

$ ./test
Segmentation fault

gdb has no stack trace, and the program prints 1234 with valgrind.

[andrewrk]

Figured it out. vfork is a function, and when the child does a return it clobber's the parent's stack. I fixed it like this:

/// This must be inline, and inline call the syscall function, because if the
/// child does a return it will clobber the parent's stack.
pub inline fn vfork() usize {
 return @inlineCall(syscall0, SYS_vfork);
}

However it's still problematic, because the example code now does this in debug builds:

x=1235

But does this in release builds:

x=1234

This is because the optimizer sees x+=1; unreachable; and then concludes that if you don't hit the if statement, the x += 1 never could have happened. The vfork breaks the compiler's understanding of control flow.

There's a better way to do this, which is to use clone followed by execve. You can see that this is how musl implements posix_spawn.

[bnoordhuis]

Sorry, I thought posix_setregid() and posix_setreuid() did the "signal other threads" dance that glibc and musl do. I see now that they only update the calling thread.

changeCurDir() and posixExecve() on the other hand allocate memory. That will be visible to other threads because the fork shares the address space with the parent.

Apropos signals, signal handlers are one reason I never used vfork() in libuv. Signals delivered to the fork will be observable in the parent when the signal handler writes to memory. And signal handlers exist for their side effects so that's probably an all too common pitfall.

[andrewrk]

Agreed, this issue is no longer to use vfork. Now it's to use clone() instead of fork() on Linux. This is how musl implements posix_spawn. That API doesn't have a way to change directory or setgid/setuid, but that's not a linux limitation; that's an API limitation.

With clone(), the memory allocations will be no problem. The parent can preallocate the memory before cloning, and then free once the child does exit or execve.

[daurnimator]

Why wouldn't you just use posix_spawn directly?