introduce std.io.poll

[andrewrk]

Motivation

I need the logic from std.ChildProcess.collectOutput for some other work I'm doing in #14647. This is my attempt to extract it into a reusable abstraction. In short summary, the usage looks like this:

// ...trychild.spawn();
varpoller=std.io.poll(gpa, enum { stdout, stderr }, .{
.stdout=child.stdout.?,
.stderr=child.stderr.?,
});
deferpoller.deinit();
while (!poller.done()) trypoller.poll();
// ...

Each stream gets a std.fifo.LinearFifo which can be accessed by e.g. poller.fifo(.stdout). You can do this inside the while loop, or you can just let it accumulate as I have done above.

Testing

Here is the code I used to test this. Before merging this PR I will break it up into parent.zig and child.zig and move it to be a standalone test.

zig build-exe test.zig
./test

conststd=@import("std");
pubfnmain() !void {
vargeneral_purpose_allocator: std.heap.GeneralPurposeAllocator(.{}) = .{};
deferif (general_purpose_allocator.deinit()) std.process.exit(1);
constgpa=general_purpose_allocator.allocator();
vararena_instance=std.heap.ArenaAllocator.init(gpa);
deferarena_instance.deinit();
constarena=arena_instance.allocator();
constargs=trystd.process.argsAlloc(arena);
if (args.len>=2) {
returnchildMain();
} else {
returnparent(gpa, arena);
}
}
fnparent(gpa: std.mem.Allocator, arena: std.mem.Allocator) !void {
constself_exe_path=trystd.fs.selfExePathAlloc(arena);
varchild=std.ChildProcess.init(&.{ self_exe_path, "child" }, gpa);
child.stdin_behavior=.Pipe;
child.stdout_behavior=.Pipe;
child.stderr_behavior=.Pipe;
trychild.spawn();
varpoller=std.io.poll(gpa, enum { stdout, stderr }, .{
.stdout=child.stdout.?,
.stderr=child.stderr.?,
});
deferpoller.deinit();
trychild.stdin.?.writeAll("the input");
child.stdin.?.close(); // send EOFchild.stdin=null;
while (!poller.done()) trypoller.poll();
constterm=trychild.wait();
switch (term) {
.Exited=>|code| {
if (code!=0) @panic("bad child exit code");
},
else=>@panic("child crash"),
}
conststdout=poller.fifo(.stdout).readableSlice(0);
conststderr=poller.fifo(.stderr).readableSlice(0);
for (0..10000) |i| {
if (!std.mem.eql(u8, stderr[i*"Garbage".len..][0.."Garbage".len], "Garbage")) {
@panic("Garbage failure");
}
if (!std.mem.eql(u8, stdout[i*"Trash".len..][0.."Trash".len], "Trash")) {
@panic("Trash failure");
}
}
}
fnchildMain() !void {
conststdout=std.io.getStdOut();
conststderr=std.io.getStdErr();
conststdin=std.io.getStdIn();
for (0..10000) |_| {
trystderr.writeAll("Garbage");
trystdout.writeAll("Trash");
}
varbuf: [1000]u8=undefined;
constamt=trystdin.readAll(&buf);
if (!std.mem.eql(u8, buf[0..amt], "the input")) {
@panic("test failure");
}
}

Merge Checklist

• Add the Windows implementation using overlapped I/O
• Rework std.ChildProcess.collectOutput to use this abstraction. Most of that code can be deleted in theory, just need to add logic to limit to max_output_bytes.
• Rework the above test file into a proper test case, probably a standalone one.

[uael]

You can consider using select instead, it is available on Macos, Linux and Windows

[ifreund]

You can consider using select instead, it is available on Macos, Linux and Windows

No, nobody should use select for anything. See the warning right at the top of the linux man page:

 WARNING: select() can monitor only file descriptors numbers that
are less than FD_SETSIZE (1024)—an unreasonably low limit for
many modern applications—and this limitation will not change.
All modern applications should instead use poll(2) or epoll(7),
which do not suffer this limitation.

[marler8997]

Just FYI, Windows uses a different ABI for fd_set which doesn't have this limit, however, it only works with sockets so not useful here.

[matu3ba]

overlapped I/O

That sounds pretty much like https://github.com/ziglang/zig/pull/14152/files#diff-e0931af874560c650fde2b1a0f92b99f2d32edb04a823cf99b6161f69f5997eaR1552-R1562 except for a socket instead of pipe.
Bear in mind that stdin can not be async without hacks and that it can be annoying for the child not introspect its calling arguments, which is why I have created the linked PR.

[marler8997]

Here's the initial windows implementation: marler8997@82ff9da

[andrewrk]

Would be nice to have the additional test, but I think this is safe to merge given how much test coverage it gets from the build system. Thanks @marler8997!