Support all operating systems, even hobby OSes, with a "bring your own OS layer" in the std lib

[andrewrk]

The std.Target.Os enum recognizes the following operating systems:

 freestanding
 ananas
 cloudabi
 dragonfly
 freebsd
 fuchsia
 ios
 kfreebsd
 linux
 lv2
 macosx
 netbsd
 openbsd
 solaris
 windows
 haiku
 minix
 rtems
 nacl
 cnk
 aix
 cuda
 nvcl
 amdhsa
 ps4
 elfiamcu
 tvos
 watchos
 mesa3d
 contiki
 amdpal
 hermit
 hurd
 wasi
 emscripten
 zen
 uefi

This list is the list of targets that LLVM supports + zen (a hobby OS not maintained for over 1 year) + UEFI.

It doesn't make sense to put every hobby OS into this list, but it does make sense to support them! It should be possible for people to take advantage of Zig's cross platform abstractions without having to get support for their hobby OS upstreamed into Zig.

I propose to do this with 2 things:

• Add other tag to std.Target.Os
• Support an OS layer struct exposed in the root source file (next to pub fn main())

This allows hobby OS developers to maintain a zig package that makes the zig standard library support their OS. Application developers could use it like this:

pub const os = @import("my_hobby_os_package");

pub fn main() void {
 // ...
}

Next, standard library abstractions will detect when they should utilize this. If the operating system is POSIX compliant, then many things will Just Work. For example, std.os.read is currently defined like this:

/// Returns the number of bytes that were read, which can be less than
/// buf.len. If 0 bytes were read, that means EOF.
/// If the application has a global event loop enabled, EAGAIN is handled
/// via the event loop. Otherwise EAGAIN results in error.WouldBlock.
pub fn read(fd: fd_t, buf: []u8) ReadError!usize {
 if (builtin.os == .windows) {
 return windows.ReadFile(fd, buf);
 }

 if (builtin.os == .wasi and !builtin.link_libc) {
 const iovs = [1]iovec{iovec{
 .iov_base = buf.ptr,
 .iov_len = buf.len,
 }};

 var nread: usize = undefined;
 switch (wasi.fd_read(fd, &iovs, iovs.len, &nread)) {
 0 => return nread,
 else => |err| return unexpectedErrno(err),
 }
 }

 while (true) {
 const rc = system.read(fd, buf.ptr, buf.len);
 switch (errno(rc)) {
 0 => return @intCast(usize, rc),
 EINTR => continue,
 EINVAL => unreachable,
 EFAULT => unreachable,
 EAGAIN => if (std.event.Loop.instance) |loop| {
 loop.waitUntilFdReadable(fd);
 continue;
 } else {
 return error.WouldBlock;
 },
 EBADF => unreachable, // Always a race condition.
 EIO => return error.InputOutput,
 EISDIR => return error.IsDir,
 ENOBUFS => return error.SystemResources,
 ENOMEM => return error.SystemResources,
 ECONNRESET => return error.ConnectionResetByPeer,
 else => |err| return unexpectedErrno(err),
 }
 }
 return index;
}

system in this refers to:

/// When linking libc, this is the C API. Otherwise, it is the OS-specific system interface.
pub const system = if (builtin.link_libc) std.c else switch (builtin.os) {
 .macosx, .ios, .watchos, .tvos => darwin,
 .freebsd => freebsd,
 .linux => linux,
 .netbsd => netbsd,
 .dragonfly => dragonfly,
 .wasi => wasi,
 .windows => windows,
 .zen => zen,
 else => struct {},
};

Here we would change else => struct{}, to look for @import("root").os if it was provided. With this modification, as long as the OS package defined all the constants (such as fd_t and EISDIR), then std.os.write would end up calling the write function from the hobby OS package, and everything Just Works.

Some abstractions may not work so smoothly; in this case there could be code that looks something like this:

fn doTheOsThing() void {
 if (std.builtin.os == .other and @hasDecl(root, "os") and @hasDecl(root.os, "doTheOsThing")) {
 return @import("root").os.doTheOsThing();
 }
}

Really, the check for the "other" OS isn't even necessary. Allowing applications to override fundamental OS functions could be useful on any operating system.

With this proposal implemented, Zig would have very near first class support for all operating systems. The main difference between upstream-recognized OSes and "other" OSes would be where the support is maintained - upstream zig std lib, or in a third party "OS layer" package.

cc @pixelherodev

[jzck]

Great idea for hobby OSes,
Could the idea be pushed even further? All operating system integration could be done via third party packages. This way upstream zig std lib wouldn't even have to know about operating system specifics.

[pixelherodev]

In theory, yes. In practice, that's not feasible - at least, not until the package manager is implemented...

I suppose with the package manager, we could extricate the OS backends, but they'd still need to be upstream, and that just makes things harder to manage.