`--watch` does not work on macOS 14.5 when saving from VSCode

[paperclover]

Zig Version

0.14.0-dev.2126+e27b4647d

Steps to Reproduce and Observed Behavior

Hardware: Apple MacBook Pro. macOS 14.5

• zig init
• zig build --watch
• Save a file using Visual Studio Code. Observe nothing happening
• Save a file using nvim. Rebuild occurs

~/code/hello $ zig build --watch
Build Summary: 5/5 steps succeeded
install success
├─ install hello success
│ └─ zig build-lib hello Debug native success 652ms MaxRSS:189M
└─ install hello success
 └─ zig build-exe hello Debug native success 671ms MaxRSS:188M
watching 23 directories, 0 processes

This does not affect other editors I've tried: Zed, macOS TextEdit, Xcode

Expected Behavior

Using a build after #21795 should have worked.

[xdBronch]

maybe a dumb question but are there edits between these saves? idk about vscode but i dont think zed actually writes anything unless it knows the buffers been changed

[paperclover]

my bad. i'm using to cmd+s always writing, and did not investigate other editors too closely besides VSCode. Zed does work if i change the contents. VSCode does not still.

I also noted some issues with creating new files, but I think that is a separate issue and might exist on linux too. Will report that accordingly

[eddiebergman]

Just adding here for MacOS:

• When using --watch and saving with zed, this does not trigger a rebuild.
• However saving with neovim does trigger a rebuild.
• Both changes are shared and visible between editors.

My guess is that different editors have different API's they use to save files, both of which can be seen by the other in terms of content, but zeds method on MacOS probably doesn't trigger the watch mechanism. I don't know what mechanism they use here.

I also used a Linux version of zed recently which did not have this problem. I'm not sure if that's due to the --watch layer of zig managing better on linux or zed's saving acts differently.

---

I went digging a bit deeper to get more context, what I can see in terms of what might actually save a file in zed are these two functions:

Zed:

https://github.com/zed-industries/zed/blob/d04800c329c380697aad7ae5b04620463c04f21e/crates/fs/src/fs.rs#L521-L562

Looking into the smol::fs::File::create, I found that it doesn't do anything unusual other than provide an asynchronous wrapper around file opening.

I also took a peek at NamedTempFile::persist() that's used in atmoic_write. It seems like it does the usual temp file write then replace, which I guess is what's causing this.

Neovim:

Hard to determine just using GitHub with C code, pretty sure they just use a plain ol' posix write()

---

No concrete evidence, just vibes, I guess zed is using the atomic_write on save, and this use of temporaries trips up zig build --watch. Hope it helps!

Edit: raised an issue with zed in case any information comes up there.

[andrewrk]

Here's the logic:

zig/lib/std/Build/Watch.zig

Lines 777 to 790 in 4de2b1e

	fn wait(w: *Watch, gpa: Allocator, timeout: Timeout) !WaitResult {
	var timespec_buffer: posix.timespec = undefined;
	var event_buffer: [100]posix.Kevent = undefined;
	var n = try posix.kevent(w.os.kq_fd, &.{}, &event_buffer, timeout.toTimespec(&timespec_buffer));
	if (n == 0) return .timeout;
	const reaction_sets = w.os.handles.items(.rs);
	var any_dirty = markDirtySteps(gpa, reaction_sets, event_buffer[0..n], false);
	timespec_buffer = .{ .sec = 0, .nsec = 0 };
	while (n == event_buffer.len) {
	n = try posix.kevent(w.os.kq_fd, &.{}, &event_buffer, &timespec_buffer);
	if (n == 0) break;
	any_dirty = markDirtySteps(gpa, reaction_sets, event_buffer[0..n], any_dirty);
	}
	return if (any_dirty) .dirty else .clean;

For starters try checking if any kqueue events are emitted at all in the atomic file update scenario.

[eddiebergman]

Some further digging:

• No events are being fired from kqueue for a write from zed. This is only for writes. New files, renaming files and deletions are all fine.
• This is not a problem with atomic_write() as I thought it was.
  • Writing a dumb C program confirms that rename(2) does actually trigger an event.

However, some new observations:

• Writing a small program that does a simple write in the very standard way of Python, Rust and C, non of these trigger an event.
• Using editors such as vi, vim, neovim and TextEdit all trigger an event.

---

Some hypothesizing in the meantime:

kqueue monitoring that is set up for Zig's watch feature is only on directories, not individual source files, i.e. you can have 3 sources files for a basic exe, ./src/main.zig, ./src/a.zig, ./src/b.zig, but only ./src will be monitored.

I couldn't find much on kqueue documentation on how that's ment to handle writes on a child of the directory, but it does manage the atomic writes, due to the fact they change what file vnodes are present in the directory. Nothing concrete but here's an SO answer (with no links -_-), which seems to align with some of the above evidence:

• Programs writing directly to files do not trigger an event.
• zed which I'm pretty confident at this point does direct file write doesn't trigger an event.
  • I boiled it down to essentially just this call rust libc call,
    whether you're in async or sync land.

  // The following code does not trigger an event.
  let file = fs::File::create(PATH).expect("Failed to create file");
  let fd = file.as_raw_fd();
  let buf = TEXT.as_bytes();
  let buf_ptr = buf.as_ptr();
  unsafe { libc::write(fd, buf_ptr as *const libc::c_void, buf.len()) };

• neovim (and family), has code dedicated to atomic writes in their buf_write(), I'm not confident it triggers but given they use swap-files and are pretty damn safe when it comes to recovery, I would guess it does.

---

If you're new to zig and/or lower level programming like me, looking for a contributor friendly issue to start, then here's some pointers I found useful:

My 10th time trying lldb is only just marginally better than the first, so print debugging ahoy. You can just edit std.Build.Watch and print debug what you need with std.log.warn. The usual std.debug.print will get consumed during zig build run --watch.

Whenever you edit your build.zig file, this will rebuild the build_runner.zig (at least for now), which in turn, uses std.Build.Watch, letting you change the print debugging. Probably a better way but oh well.

In terms of places to look, there's also fn update() where the actual build steps are passed, and registered to kqueue. These directories are the ones being monitored.

fn update(w: *Watch, gpa: Allocator, steps: []const *Step) !void {
 const handles = &w.os.handles;
 for (steps) |step| {
 for (step.inputs.table.keys(), step.inputs.table.values()) |path, *files| {
 const reaction_set = rs: {
 const gop = try w.dir_table.getOrPut(gpa, path);

 //
 std.log.warn("index {d} {}\n", .{ gop.index, path });

# warning: index 0 src
# warning: index 1 /path/.zvm/master/lib
# warning: index 2 /path/.zvm/master/lib/std
# warning: index 3 /path/.cache/zig/b/cd3e39922319d072df006eea662bc270
# ... And 19 others

If you add some prints to the wait(), I'm not sure why but usually I see the same event being fired off multiple times (3-4). Here's just a sample. For meaning of numbers, see std.c and the enum values passed to Kevent struct.

// .ident = file_descriptor
// .filter = VNode
// .flags = ADD to queue, ENABLE it, CLEAR from event buffer when triggered
// .fflags = 2 written to the vnode (i.e. some change in directory contents)
// .udata = 0, some data zig tells kqueue to associate to each registered event type, in this case, the index above. 
warning: 1 events: { c.Kevent__struct_5811{ .ident = 9, .filter = -4, .flags = 37, .fflags = 2, .data = 0, .udata = 0 } }
warning: 1 events: { c.Kevent__struct_5811{ .ident = 9, .filter = -4, .flags = 37, .fflags = 2, .data = 0, .udata = 0 } }
warning: 1 events: { c.Kevent__struct_5811{ .ident = 9, .filter = -4, .flags = 37, .fflags = 2, .data = 0, .udata = 0 } }

[andrewrk]

The watch system would not want to respond to writes; the file may be open for many writes before it is closed. In this case what the watching system is interested in is when a file in a watched directory that was open for writing is closed.

For example, FreeBSD has NOTE.CLOSE_WRITE. macOS appears to not have this feature.

Looks like the NOTE.WRITE there is actually doing nothing since it's a directory fd, not a file fd. In order to fix this issue it seems a kevent would need to be created for every watched file individually, which may not be a viable solution, because it may trigger EMFILE. It would also be significantly less efficient than what can be achieved by the other operating system APIs.

This may be something that we resolve by documenting the limitation rather than trying to work around it.

[eddiebergman]

The watch system would not want to respond to writes; the file may be open for many writes before it is closed. In this case what the watching system is interested in is when a file in a watched directory that was open for writing is closed.

Makes a lot of sense. I believe this is what was going on in the test programs I made as well as zed, i.e. open, write, close. I should have been a little clearer there.

Taking a quick look at what else is out there for watch, I only really found two solutions:

• Using kqueue and eating the cost
• Using FSEvents

It seems FSEvent's recommendation for this kind of work (See Building a Directory Hierarchy Snapshot in the linked fsevents docs) is non-trivial and with some quick looking around, it always seems to come with caveats which are likely a maintenance burden. (ref1, ref2).

This may be something that we resolve by documenting the limitation rather than trying to work around it.

That's probably the best course of action.

Sorry if this was all known to you before and seems obvious, just hoping the documenting of it helps someone else who comes along.

[andrewrk]

Sorry if this was all known to you before and seems obvious, just hoping the documenting of it helps someone else who comes along.

No need to apologize! Thanks for looking into it and writing your notes.