std: fuck usingnamespace

[mlugg]

see title

[Rexicon226]

ci failure: #19089

[mlugg]

Whoops, I accidentally flipped the conditions in that commit.

[rofrol]

Why fuck?

[mlugg]

The PR title is kind of a joke, but: the usingnamespace construct is problematic for implementing incremental compilation, and broadly speaking, tends to result in worse, less readable code. It may be removed from the language in the future. We are replacing all uses of it in std, to clean up code as well as help us evaluate potential use cases for the feature.

[mlugg]

@mitchellh and @kprotty (best ping determined from blame), I notice that libxev and tigerbeetle are two notable consumers of our io_uring API; do the changes I've made here look reasonable? It's just some renames and small reshuffles.

• linux.IO_Uring --> linux.IoUring to align with naming conventions
• All io_uring_prep_xyz are now prep_xyz methods on io_uring_sqe (I think this is only an internal detail?)
• SubmissionQueue and CompletionQueue are namespaced under IoUring

[Inve1951]

I'd argue that io_uring_sqe should be named IoUringSqe (or IoUring.SubmissionQueueEntry) and its prep* methods should return an instance of it instead of consuming a pointer.

Edit: And then maybe instead of calling them prep* calling them init*.

[mlugg]

I agree that would probably make sense, but it can be a future enhancement. For now, I'm more concerned with validating that this new API - mainly motivated by the removal of this usingnamespace - is reasonable for end users.

[Jarred-Sumner]

This is probably the wrong issue to mention this but I’d be really sad if usingnamespace is removed from Zig without an adequate replacement. We use it frequently in Bun for interface boundaries between different parts of the codebase and for ergonomically mixing handwritten code with generated code. I definitely agree it’s not perfect, but removing it without addressing any of the problems it solves would not be better for Zig’s users.

[kprotty]

The io_uring refactors seem fine (namespacing io_uring_sqe under IoUring also seems appropriate given the other changes). The potential future usingnamespace replacements are verbose too, but in the end doable.

[mlugg]

Okay, I'm going to go ahead with this one - the breakages are pretty trivial, and I have the Protty Stamp Of Approval (tm). I agree with the other proposed changes to the io_uring API, but they can be handled at a later date.

[mitchellh]

Re: libxev. I'm sure it'll be fine 😄 I'll report back if I run into any issues upgrading to nightly.

[silversquirl]

@Jarred-Sumner For mixing generated and handwritten code, you can make your generator use a file like this as a template:

pubconstFoo=struct {
// GENERATED:Foopubfnfoo(self: Foo) void {
// ...
}
};
pubconstBar=struct {
// GENERATED:Barpubfnbar(self: Bar) void {
// ...
}
};

Instead of outputting full structs, it simply replaces the // GENERATED: comments. That way you can write whatever code you like around the generated code, without needing to use usingnamespace

[Jarred-Sumner]

@Jarred-Sumner For mixing generated and handwritten code, you can make your generator use a file like this as a template. Instead of outputting full structs, it simply replaces the // GENERATED: comments. That way you can write whatever code you like around the generated code, without needing to use usingnamespace

While yes it is possible to do that, it's hard to see how that is better than usingnamespace. This would break editor autocomplete and it adds a layer of indirection (the file itself cannot be imported, instead all usages must import the generated file) which would need to be explained and re-explained each time someone imports the non-generated file

[perillo]

@mlugg Commit 17f83ac removed IndexedMap, IndexedArray, IndexedSet. What about moving the code to https://github.com/ziglang/std-lib-orphanage?

[matklad]

This would break editor autocomplete

As someone who has spend a fair amount of time building editor auto-complete for Rust, I am 0.85 sure that removal of usingnamespace, in the long run, would be a boon for autocomplete. Dealing with star imports, and, more generally, with any kind of unbounded identifier search is a major complexity and performance pain point for IDE tooling.

That being said, yes, removing usingnamespace would make code-generation use-case worse. The question is about the tradeoffs between convenience of source-code generation on one side of the scale, and a faster, simpler & more powerful compiler on the other one.

The source code generation problem doesn't seem insurmountable to me. I'd say that any of the following solutions would work okay, if not perfect, in practice:

• generate code to foo_generated.zig and require importing generated files at the call-site.

• make sure that the entire code is generated. So, you'll have

  ./foo.zig
  ./foo_helpers.zig
  

  Then, foo.zig is the generated file, the entirety of file is generated, but the generated code also contains const helpers = @import("foo_helpers.zig") and calls functions from helpers when appropriate. The hand-written code goes to the helpers.zig file, but that's a private implementation detail, the call-sites always just import foo. Just how you generate the entirety of foo might be different, but the template file silversquirl is one option (as long as all the actual hand-written logic lives in the helpers

• manually delegate all symbols. So, there's foo.zig and foo_generated.zig, and the former has hand-written code like const my_func = @import("./foo_generated.zig").my_func. Writing delegation by hand is tiresome, but you'll only need to do this twice.

  more generally, unless the generated/hand-written split is roughly even, one of the previous two solutions (generate the entire API vs manual delegation) should work fine

• if there's a bunch of generated code, but also a bunch of hand-written code, one can consider self-modifying code: https://matklad.github.io/2022/03/26/self-modifying-code.html#Self-Modifying-Code-1

  This is similar to the template approach, except that it looks more like

  pubconstFoo=struct {
  // GENERATEDSTART:Foopubfnmy_generated_func(self: Foo) void { ... }
  // GENERATEDEND:Foopubfnfoo(self: Foo) void {
  // ...
  }
  };

  the code generator then just updates the file in-place

Again, these all are work-around, and usingnamespace solves this particular advanced use-case better, but it also imposes diffused costs on more-or-less every other use-case.

[Jarred-Sumner]

As someone who has spend a fair amount of time building editor auto-complete for Rust, I am 0.85 sure that removal of usingnamespace, in the long run, would be a boon for autocomplete. Dealing with star imports, and, more generally, with any kind of unbounded identifier search is a major complexity and performance pain point for IDE tooling.

You certainly know more about editor autocomplete than I do and that makes sense.

I'm more concerned by the mixin-shaped hole removing usingnamespace would leave (interface boundaries). Rust-like Traits, Go-like interfaces, or TypeScript-like interfaces (semantic analysis validates, but compiles equivalently to anytype) would be an improvement over usingnamespace, but removing it without replacing it with something else would leave us worse off than the status quo

[matklad]

Yeah, "mixings" argument feels stronger to me than "source code generation" one! I don't have any particular insights here though!

[blblack]

@mlugg I tried to naively remove usingnamespace from my PR in #19203 and of course broke the smoke test I had included there. The general pattern I'm looking at here is: without usingnamespace, if you want to test a libc/sys -call interface that only exists on a subset of platforms, what's the new correct way to do that?

Before I was relying on the platform-conditional declaration via usingnamespace and then using @hasDecl in the test block to skip the test on non-supporting platforms.

I suppose one answer would be to move the list of supporting platforms to a conditional over the test code, but that doesn't quite feel right, either. If another new platform is added which happens to have this call, the plumbing would work thanks to the universal declaration, but the tests would be skipping this new platform and nobody may notice or care.

I also naively tried something like this:

pub const has_mlockall = switch (native_os) {
.linux, .freebsd, .netbsd, .openbsd, .dragonfly, .solaris => true,
else => false,
}; pub const mlockall = if (has_mlockall) struct {
extern fn mlockall(flags: c_int) c_int;
}.mlockall else undefined;

... but this still results in the declaration existing, it just happens to have an undefined value.

[blblack]

I guess the other reasonable alternative would be to stop trying to be so DRY in lib/std/c.zig with these platform conditionals and just put the redundant declarations in the various lib/std/c/foo.zig files (but then that itself currently still relies on a remaining use of usingnamespace to conditionally import them).

[blblack]

Hmm, how horrible would it be for the language to just have an explicit way to say that a block doesn't create a new scope, and thus is only useful for condition/looping/etc. For example:

// normal block, x is in a new sub-scope
if (foo == bar) {
pub const x = undefined;
}
// scope-less block, serves the purposes of "if" conditional,
// but creates x in containing scope.
if (foo == bar) <{
pub const x = undefined;
}>

[nektro]

that's precisely what usingnamespace does

[blblack]

that's precisely what usingnamespace does

Kinda, but not precisely. usingnamespace expects the operand to be a struct, union, or enum (or similarly, opaque type). What I'm suggesting above is just "this block of code is a block for flow-control purposes, but does not introduce a new sub-level of naming scope".