mingw: Use aro instead of clang for preprocessing import libs

[ehaas]

Based on manual testing, this produces the same output for kernel32.def when compared to clang, other than minor whitespace differences (clang puts a newline at the beginning, and a trailing space on one line which aro does not)

Testing steps:

1. Clear zig cache
2. Create empty test.c
3. With status-quo zig: zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print a zig clang line to stderr; the output path is the final argument
4. With this branch: ./zig-out/bin/zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print output path: <ZIG_CACHE_PATH>/o/<SOME HASH>/kernel32.def
5. Diff the two files
6. Available targets are x86_64-windows-gnu, x86-windows-gnu, arm-windows-gnu, aarch64-windows-gnu

I manually tested error handling by editing kernel32.def.in to have an invalid preprocessing directive and got the following:

zig-out/lib/zig/libc/mingw/lib-common/kernel32.def.in:1:1: error: invalid preprocessing directive
#foo
^
1 error generated.
error: unable to generate DLL import .lib file for kernel32: AroPreprocessorFailed

Closes#17753

[andrewrk]

I suggest to put a check for build_options.only_core_functionality to prevent Aro from being compiled into zig1.wasm or zig2.c during the bootstrap process. This will also fix the current CI failures.

[ehaas]

Is there a way to test this locally if I don't have access to a windows machine? I don't want to spam the CI with attempts, since I'm not totally sure where the check should go. It seems like the function is being used, since we're hitting that panic - does that mean that if build_options.only_core_functionality is true, we should use the existing zig clang solution, and otherwise use the new aro one?

[andrewrk]

Yes, here is a way to test it:

1. put @compileError() somewhere inside aro
2. zig build -Donly-c

If you get the compile error, it's incorrectly including aro in the build.

The check goes at the top of buildImportLib:

if (build_options.only_core_functionality) @panic("building import libs not included in core functionality");

[andrewrk]

Oops, I gave you some bad advice. It should be checking only_c instead of only_core_functionality.

We don't want the logic to be part of zig1.wasm, however we do want the logic to be part of zig2.c.

The use case for only_core_functionality is for example package fetching. We certainly do not need zig2 to have the package manager logic inside of it.

[ehaas]

Ok, if I add that I assume it means it's also safe for me to remove the if (builtin.zig_backend == .stage2_c) @panic("the CBE cannot compile Aro yet!"); from later in the function, right before I import aro?

[andrewrk]

Ah, I forgot about this. I'll open another issue for making zig support writing import libraries from def files.

[andrewrk]

#17807

[ehaas]

@Vexu I will need your help integrating the arocc's GenerateDef changes with zig's build.zig - I'm not sure how to add the def files to the aro module.

[Vexu]

I don't think there's going to be an easy way to do so because of the bootstrapping process. I'd just copy the generated files from the cache and replace the def files.

[Vexu]

One possible option would be to add a stubbed def file and then in CMake add that as the module for the def files for the purpose of bootstrapping. For building them regularly it should be easy enough to copy some parts from Aro's build.zig.

This should enable bootstrapping:

pubfnwith(comptimeProperties: type) type {
returnstruct {
tag: Tag=@enumFromInt(0),
properties: Properties=undefined,
pubconstmax_param_count=1;
pubconstlongest_name=0;
pubconstdata= [_]@This(){.{}};
pubinlinefnfromName(_: []constu8) ?@This() {
returnif (@hasField(Properties, "declspec")) nullelse .{};
}
pubfnnameFromUniqueIndex(_: u16, _: []u8) []u8 {
return"";
}
pubfnuniqueIndex(_: []constu8) ?u16 {
returnnull;
}
pubconstTag=enum(u16) { _ };
pubfnnameFromTag(_: Tag) NameBuf {
return .{};
}
pubfntagFromName(name: []constu8) ?Tag {
return@enumFromInt(name.len);
}
pubconstNameBuf=struct {
pubfnspan(_: *constNameBuf) []constu8 {
return"";
}
};
};
}

[Vexu]

I forgot about the "file exists in multiple modules" error again and made the stub work for both. It could be simplified for them individually but this works too.

I simplified the names stub while fixing the build error.

[ehaas]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files. I fixed it with the following diff:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index dcbddc476..d5fd9b4b1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -647,8 +647,11 @@ set(ZIG_STAGE2_SOURCES
"${CMAKE_SOURCE_DIR}/src/windows_sdk.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_builtins.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_attributes.zig"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Attribute/names.def"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Builtins/Builtin.def"
)
+
if(MSVC)
set(MSVC_DIA_SDK_DIR "$ENV{VSINSTALLDIR}DIA SDK")
if(IS_DIRECTORY ${MSVC_DIA_SDK_DIR})
@@ -947,3 +950,10 @@ install(CODE "set(ZIG_BUILD_ARGS \"${ZIG_BUILD_ARGS}\")")
install(CODE "set(CMAKE_INSTALL_PREFIX \"${CMAKE_INSTALL_PREFIX}\")")
install(CODE "set(CMAKE_SOURCE_DIR \"${CMAKE_SOURCE_DIR}\")")
install(SCRIPT "${CMAKE_SOURCE_DIR}/cmake/install.cmake")
+
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Attribute/names.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Attribute)
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Builtins/Builtin.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Builtins)

I'm not sure if this is the correct fix; I also did this without your changes to this branch or to aro from today.

[andrewrk]

Please don't add any additional logic to CMakeLists.txt. Zig should depend on Aro only as a set of zig files. I don't want to add any Aro-related build system logic to zig's build process.

[Vexu]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files.

I think the issue was Vexu/arocc@9f4c28a

[ehaas]

If we want to completely eliminate the CMakeLists.txt changes (so, no stub modules or anything) I have a local patch that does that by replacing the .def file contents with the generated versions. It succeeds at the bootstrap + stage4 build. Let me know if that is desired and I can push it.

[andrewrk]

Thanks @ehaas. I think this is fine - passing a few more args to building zig2.c is OK, I just didn't want to add any more steps to the process. For example, to update #17892 after merging this PR, I will only need to adjust a few command line args, but not introduce more logic into bootstrap.c.

[ehaas]

Based on manual testing, this produces the same output for kernel32.def when compared to clang, other than minor whitespace differences (clang puts a newline at the beginning, and a trailing space on one line which aro does not)

Testing steps:

1. Clear zig cache
2. Create empty test.c
3. With status-quo zig: zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print a zig clang line to stderr; the output path is the final argument
4. With this branch: ./zig-out/bin/zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print output path: <ZIG_CACHE_PATH>/o/<SOME HASH>/kernel32.def
5. Diff the two files
6. Available targets are x86_64-windows-gnu, x86-windows-gnu, arm-windows-gnu, aarch64-windows-gnu

I manually tested error handling by editing kernel32.def.in to have an invalid preprocessing directive and got the following:

zig-out/lib/zig/libc/mingw/lib-common/kernel32.def.in:1:1: error: invalid preprocessing directive
#foo
^
1 error generated.
error: unable to generate DLL import .lib file for kernel32: AroPreprocessorFailed

Closes#17753

[andrewrk]

I suggest to put a check for build_options.only_core_functionality to prevent Aro from being compiled into zig1.wasm or zig2.c during the bootstrap process. This will also fix the current CI failures.

[ehaas]

Is there a way to test this locally if I don't have access to a windows machine? I don't want to spam the CI with attempts, since I'm not totally sure where the check should go. It seems like the function is being used, since we're hitting that panic - does that mean that if build_options.only_core_functionality is true, we should use the existing zig clang solution, and otherwise use the new aro one?

[andrewrk]

Yes, here is a way to test it:

1. put @compileError() somewhere inside aro
2. zig build -Donly-c

If you get the compile error, it's incorrectly including aro in the build.

The check goes at the top of buildImportLib:

if (build_options.only_core_functionality) @panic("building import libs not included in core functionality");

[andrewrk]

Oops, I gave you some bad advice. It should be checking only_c instead of only_core_functionality.

We don't want the logic to be part of zig1.wasm, however we do want the logic to be part of zig2.c.

The use case for only_core_functionality is for example package fetching. We certainly do not need zig2 to have the package manager logic inside of it.

[ehaas]

Ok, if I add that I assume it means it's also safe for me to remove the if (builtin.zig_backend == .stage2_c) @panic("the CBE cannot compile Aro yet!"); from later in the function, right before I import aro?

[andrewrk]

Ah, I forgot about this. I'll open another issue for making zig support writing import libraries from def files.

[andrewrk]

#17807

[ehaas]

@Vexu I will need your help integrating the arocc's GenerateDef changes with zig's build.zig - I'm not sure how to add the def files to the aro module.

[Vexu]

I don't think there's going to be an easy way to do so because of the bootstrapping process. I'd just copy the generated files from the cache and replace the def files.

[Vexu]

One possible option would be to add a stubbed def file and then in CMake add that as the module for the def files for the purpose of bootstrapping. For building them regularly it should be easy enough to copy some parts from Aro's build.zig.

This should enable bootstrapping:

pubfnwith(comptimeProperties: type) type {
returnstruct {
tag: Tag=@enumFromInt(0),
properties: Properties=undefined,
pubconstmax_param_count=1;
pubconstlongest_name=0;
pubconstdata= [_]@This(){.{}};
pubinlinefnfromName(_: []constu8) ?@This() {
returnif (@hasField(Properties, "declspec")) nullelse .{};
}
pubfnnameFromUniqueIndex(_: u16, _: []u8) []u8 {
return"";
}
pubfnuniqueIndex(_: []constu8) ?u16 {
returnnull;
}
pubconstTag=enum(u16) { _ };
pubfnnameFromTag(_: Tag) NameBuf {
return .{};
}
pubfntagFromName(name: []constu8) ?Tag {
return@enumFromInt(name.len);
}
pubconstNameBuf=struct {
pubfnspan(_: *constNameBuf) []constu8 {
return"";
}
};
};
}

[Vexu]

I forgot about the "file exists in multiple modules" error again and made the stub work for both. It could be simplified for them individually but this works too.

I simplified the names stub while fixing the build error.

[ehaas]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files. I fixed it with the following diff:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index dcbddc476..d5fd9b4b1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -647,8 +647,11 @@ set(ZIG_STAGE2_SOURCES
"${CMAKE_SOURCE_DIR}/src/windows_sdk.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_builtins.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_attributes.zig"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Attribute/names.def"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Builtins/Builtin.def"
)
+
if(MSVC)
set(MSVC_DIA_SDK_DIR "$ENV{VSINSTALLDIR}DIA SDK")
if(IS_DIRECTORY ${MSVC_DIA_SDK_DIR})
@@ -947,3 +950,10 @@ install(CODE "set(ZIG_BUILD_ARGS \"${ZIG_BUILD_ARGS}\")")
install(CODE "set(CMAKE_INSTALL_PREFIX \"${CMAKE_INSTALL_PREFIX}\")")
install(CODE "set(CMAKE_SOURCE_DIR \"${CMAKE_SOURCE_DIR}\")")
install(SCRIPT "${CMAKE_SOURCE_DIR}/cmake/install.cmake")
+
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Attribute/names.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Attribute)
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Builtins/Builtin.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Builtins)

I'm not sure if this is the correct fix; I also did this without your changes to this branch or to aro from today.

[andrewrk]

Please don't add any additional logic to CMakeLists.txt. Zig should depend on Aro only as a set of zig files. I don't want to add any Aro-related build system logic to zig's build process.

[Vexu]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files.

I think the issue was Vexu/arocc@9f4c28a

[ehaas]

If we want to completely eliminate the CMakeLists.txt changes (so, no stub modules or anything) I have a local patch that does that by replacing the .def file contents with the generated versions. It succeeds at the bootstrap + stage4 build. Let me know if that is desired and I can push it.

[andrewrk]

Thanks @ehaas. I think this is fine - passing a few more args to building zig2.c is OK, I just didn't want to add any more steps to the process. For example, to update #17892 after merging this PR, I will only need to adjust a few command line args, but not introduce more logic into bootstrap.c.

[ehaas]

Based on manual testing, this produces the same output for kernel32.def when compared to clang, other than minor whitespace differences (clang puts a newline at the beginning, and a trailing space on one line which aro does not)

Testing steps:

1. Clear zig cache
2. Create empty test.c
3. With status-quo zig: zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print a zig clang line to stderr; the output path is the final argument
4. With this branch: ./zig-out/bin/zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print output path: <ZIG_CACHE_PATH>/o/<SOME HASH>/kernel32.def
5. Diff the two files
6. Available targets are x86_64-windows-gnu, x86-windows-gnu, arm-windows-gnu, aarch64-windows-gnu

I manually tested error handling by editing kernel32.def.in to have an invalid preprocessing directive and got the following:

zig-out/lib/zig/libc/mingw/lib-common/kernel32.def.in:1:1: error: invalid preprocessing directive
#foo
^
1 error generated.
error: unable to generate DLL import .lib file for kernel32: AroPreprocessorFailed

Closes#17753

[andrewrk]

I suggest to put a check for build_options.only_core_functionality to prevent Aro from being compiled into zig1.wasm or zig2.c during the bootstrap process. This will also fix the current CI failures.

[ehaas]

Is there a way to test this locally if I don't have access to a windows machine? I don't want to spam the CI with attempts, since I'm not totally sure where the check should go. It seems like the function is being used, since we're hitting that panic - does that mean that if build_options.only_core_functionality is true, we should use the existing zig clang solution, and otherwise use the new aro one?

[andrewrk]

Yes, here is a way to test it:

1. put @compileError() somewhere inside aro
2. zig build -Donly-c

If you get the compile error, it's incorrectly including aro in the build.

The check goes at the top of buildImportLib:

if (build_options.only_core_functionality) @panic("building import libs not included in core functionality");

[andrewrk]

Oops, I gave you some bad advice. It should be checking only_c instead of only_core_functionality.

We don't want the logic to be part of zig1.wasm, however we do want the logic to be part of zig2.c.

The use case for only_core_functionality is for example package fetching. We certainly do not need zig2 to have the package manager logic inside of it.

[ehaas]

Ok, if I add that I assume it means it's also safe for me to remove the if (builtin.zig_backend == .stage2_c) @panic("the CBE cannot compile Aro yet!"); from later in the function, right before I import aro?

[andrewrk]

Ah, I forgot about this. I'll open another issue for making zig support writing import libraries from def files.

[andrewrk]

#17807

[ehaas]

@Vexu I will need your help integrating the arocc's GenerateDef changes with zig's build.zig - I'm not sure how to add the def files to the aro module.

[Vexu]

I don't think there's going to be an easy way to do so because of the bootstrapping process. I'd just copy the generated files from the cache and replace the def files.

[Vexu]

One possible option would be to add a stubbed def file and then in CMake add that as the module for the def files for the purpose of bootstrapping. For building them regularly it should be easy enough to copy some parts from Aro's build.zig.

This should enable bootstrapping:

pubfnwith(comptimeProperties: type) type {
returnstruct {
tag: Tag=@enumFromInt(0),
properties: Properties=undefined,
pubconstmax_param_count=1;
pubconstlongest_name=0;
pubconstdata= [_]@This(){.{}};
pubinlinefnfromName(_: []constu8) ?@This() {
returnif (@hasField(Properties, "declspec")) nullelse .{};
}
pubfnnameFromUniqueIndex(_: u16, _: []u8) []u8 {
return"";
}
pubfnuniqueIndex(_: []constu8) ?u16 {
returnnull;
}
pubconstTag=enum(u16) { _ };
pubfnnameFromTag(_: Tag) NameBuf {
return .{};
}
pubfntagFromName(name: []constu8) ?Tag {
return@enumFromInt(name.len);
}
pubconstNameBuf=struct {
pubfnspan(_: *constNameBuf) []constu8 {
return"";
}
};
};
}

[Vexu]

I forgot about the "file exists in multiple modules" error again and made the stub work for both. It could be simplified for them individually but this works too.

I simplified the names stub while fixing the build error.

[ehaas]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files. I fixed it with the following diff:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index dcbddc476..d5fd9b4b1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -647,8 +647,11 @@ set(ZIG_STAGE2_SOURCES
"${CMAKE_SOURCE_DIR}/src/windows_sdk.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_builtins.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_attributes.zig"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Attribute/names.def"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Builtins/Builtin.def"
)
+
if(MSVC)
set(MSVC_DIA_SDK_DIR "$ENV{VSINSTALLDIR}DIA SDK")
if(IS_DIRECTORY ${MSVC_DIA_SDK_DIR})
@@ -947,3 +950,10 @@ install(CODE "set(ZIG_BUILD_ARGS \"${ZIG_BUILD_ARGS}\")")
install(CODE "set(CMAKE_INSTALL_PREFIX \"${CMAKE_INSTALL_PREFIX}\")")
install(CODE "set(CMAKE_SOURCE_DIR \"${CMAKE_SOURCE_DIR}\")")
install(SCRIPT "${CMAKE_SOURCE_DIR}/cmake/install.cmake")
+
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Attribute/names.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Attribute)
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Builtins/Builtin.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Builtins)

I'm not sure if this is the correct fix; I also did this without your changes to this branch or to aro from today.

[andrewrk]

Please don't add any additional logic to CMakeLists.txt. Zig should depend on Aro only as a set of zig files. I don't want to add any Aro-related build system logic to zig's build process.

[Vexu]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files.

I think the issue was Vexu/arocc@9f4c28a

[ehaas]

If we want to completely eliminate the CMakeLists.txt changes (so, no stub modules or anything) I have a local patch that does that by replacing the .def file contents with the generated versions. It succeeds at the bootstrap + stage4 build. Let me know if that is desired and I can push it.

[andrewrk]

Thanks @ehaas. I think this is fine - passing a few more args to building zig2.c is OK, I just didn't want to add any more steps to the process. For example, to update #17892 after merging this PR, I will only need to adjust a few command line args, but not introduce more logic into bootstrap.c.

[ehaas]

Based on manual testing, this produces the same output for kernel32.def when compared to clang, other than minor whitespace differences (clang puts a newline at the beginning, and a trailing space on one line which aro does not)

Testing steps:

1. Clear zig cache
2. Create empty test.c
3. With status-quo zig: zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print a zig clang line to stderr; the output path is the final argument
4. With this branch: ./zig-out/bin/zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print output path: <ZIG_CACHE_PATH>/o/<SOME HASH>/kernel32.def
5. Diff the two files
6. Available targets are x86_64-windows-gnu, x86-windows-gnu, arm-windows-gnu, aarch64-windows-gnu

I manually tested error handling by editing kernel32.def.in to have an invalid preprocessing directive and got the following:

zig-out/lib/zig/libc/mingw/lib-common/kernel32.def.in:1:1: error: invalid preprocessing directive
#foo
^
1 error generated.
error: unable to generate DLL import .lib file for kernel32: AroPreprocessorFailed

Closes#17753

[andrewrk]

I suggest to put a check for build_options.only_core_functionality to prevent Aro from being compiled into zig1.wasm or zig2.c during the bootstrap process. This will also fix the current CI failures.

[ehaas]

Is there a way to test this locally if I don't have access to a windows machine? I don't want to spam the CI with attempts, since I'm not totally sure where the check should go. It seems like the function is being used, since we're hitting that panic - does that mean that if build_options.only_core_functionality is true, we should use the existing zig clang solution, and otherwise use the new aro one?

[andrewrk]

Yes, here is a way to test it:

1. put @compileError() somewhere inside aro
2. zig build -Donly-c

If you get the compile error, it's incorrectly including aro in the build.

The check goes at the top of buildImportLib:

if (build_options.only_core_functionality) @panic("building import libs not included in core functionality");

[andrewrk]

Oops, I gave you some bad advice. It should be checking only_c instead of only_core_functionality.

We don't want the logic to be part of zig1.wasm, however we do want the logic to be part of zig2.c.

The use case for only_core_functionality is for example package fetching. We certainly do not need zig2 to have the package manager logic inside of it.

[ehaas]

Ok, if I add that I assume it means it's also safe for me to remove the if (builtin.zig_backend == .stage2_c) @panic("the CBE cannot compile Aro yet!"); from later in the function, right before I import aro?

[andrewrk]

Ah, I forgot about this. I'll open another issue for making zig support writing import libraries from def files.

[andrewrk]

#17807

[ehaas]

@Vexu I will need your help integrating the arocc's GenerateDef changes with zig's build.zig - I'm not sure how to add the def files to the aro module.

[Vexu]

I don't think there's going to be an easy way to do so because of the bootstrapping process. I'd just copy the generated files from the cache and replace the def files.

[Vexu]

One possible option would be to add a stubbed def file and then in CMake add that as the module for the def files for the purpose of bootstrapping. For building them regularly it should be easy enough to copy some parts from Aro's build.zig.

This should enable bootstrapping:

pubfnwith(comptimeProperties: type) type {
returnstruct {
tag: Tag=@enumFromInt(0),
properties: Properties=undefined,
pubconstmax_param_count=1;
pubconstlongest_name=0;
pubconstdata= [_]@This(){.{}};
pubinlinefnfromName(_: []constu8) ?@This() {
returnif (@hasField(Properties, "declspec")) nullelse .{};
}
pubfnnameFromUniqueIndex(_: u16, _: []u8) []u8 {
return"";
}
pubfnuniqueIndex(_: []constu8) ?u16 {
returnnull;
}
pubconstTag=enum(u16) { _ };
pubfnnameFromTag(_: Tag) NameBuf {
return .{};
}
pubfntagFromName(name: []constu8) ?Tag {
return@enumFromInt(name.len);
}
pubconstNameBuf=struct {
pubfnspan(_: *constNameBuf) []constu8 {
return"";
}
};
};
}

[Vexu]

I forgot about the "file exists in multiple modules" error again and made the stub work for both. It could be simplified for them individually but this works too.

I simplified the names stub while fixing the build error.

[ehaas]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files. I fixed it with the following diff:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index dcbddc476..d5fd9b4b1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -647,8 +647,11 @@ set(ZIG_STAGE2_SOURCES
"${CMAKE_SOURCE_DIR}/src/windows_sdk.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_builtins.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_attributes.zig"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Attribute/names.def"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Builtins/Builtin.def"
)
+
if(MSVC)
set(MSVC_DIA_SDK_DIR "$ENV{VSINSTALLDIR}DIA SDK")
if(IS_DIRECTORY ${MSVC_DIA_SDK_DIR})
@@ -947,3 +950,10 @@ install(CODE "set(ZIG_BUILD_ARGS \"${ZIG_BUILD_ARGS}\")")
install(CODE "set(CMAKE_INSTALL_PREFIX \"${CMAKE_INSTALL_PREFIX}\")")
install(CODE "set(CMAKE_SOURCE_DIR \"${CMAKE_SOURCE_DIR}\")")
install(SCRIPT "${CMAKE_SOURCE_DIR}/cmake/install.cmake")
+
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Attribute/names.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Attribute)
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Builtins/Builtin.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Builtins)

I'm not sure if this is the correct fix; I also did this without your changes to this branch or to aro from today.

[andrewrk]

Please don't add any additional logic to CMakeLists.txt. Zig should depend on Aro only as a set of zig files. I don't want to add any Aro-related build system logic to zig's build process.

[Vexu]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files.

I think the issue was Vexu/arocc@9f4c28a

[ehaas]

If we want to completely eliminate the CMakeLists.txt changes (so, no stub modules or anything) I have a local patch that does that by replacing the .def file contents with the generated versions. It succeeds at the bootstrap + stage4 build. Let me know if that is desired and I can push it.

[andrewrk]

Thanks @ehaas. I think this is fine - passing a few more args to building zig2.c is OK, I just didn't want to add any more steps to the process. For example, to update #17892 after merging this PR, I will only need to adjust a few command line args, but not introduce more logic into bootstrap.c.

[ehaas]

Based on manual testing, this produces the same output for kernel32.def when compared to clang, other than minor whitespace differences (clang puts a newline at the beginning, and a trailing space on one line which aro does not)

Testing steps:

1. Clear zig cache
2. Create empty test.c
3. With status-quo zig: zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print a zig clang line to stderr; the output path is the final argument
4. With this branch: ./zig-out/bin/zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print output path: <ZIG_CACHE_PATH>/o/<SOME HASH>/kernel32.def
5. Diff the two files
6. Available targets are x86_64-windows-gnu, x86-windows-gnu, arm-windows-gnu, aarch64-windows-gnu

I manually tested error handling by editing kernel32.def.in to have an invalid preprocessing directive and got the following:

zig-out/lib/zig/libc/mingw/lib-common/kernel32.def.in:1:1: error: invalid preprocessing directive
#foo
^
1 error generated.
error: unable to generate DLL import .lib file for kernel32: AroPreprocessorFailed

Closes#17753

[andrewrk]

I suggest to put a check for build_options.only_core_functionality to prevent Aro from being compiled into zig1.wasm or zig2.c during the bootstrap process. This will also fix the current CI failures.

[ehaas]

Is there a way to test this locally if I don't have access to a windows machine? I don't want to spam the CI with attempts, since I'm not totally sure where the check should go. It seems like the function is being used, since we're hitting that panic - does that mean that if build_options.only_core_functionality is true, we should use the existing zig clang solution, and otherwise use the new aro one?

[andrewrk]

Yes, here is a way to test it:

1. put @compileError() somewhere inside aro
2. zig build -Donly-c

If you get the compile error, it's incorrectly including aro in the build.

The check goes at the top of buildImportLib:

if (build_options.only_core_functionality) @panic("building import libs not included in core functionality");

[andrewrk]

Oops, I gave you some bad advice. It should be checking only_c instead of only_core_functionality.

We don't want the logic to be part of zig1.wasm, however we do want the logic to be part of zig2.c.

The use case for only_core_functionality is for example package fetching. We certainly do not need zig2 to have the package manager logic inside of it.

[ehaas]

Ok, if I add that I assume it means it's also safe for me to remove the if (builtin.zig_backend == .stage2_c) @panic("the CBE cannot compile Aro yet!"); from later in the function, right before I import aro?

[andrewrk]

Ah, I forgot about this. I'll open another issue for making zig support writing import libraries from def files.

[andrewrk]

#17807

[ehaas]

@Vexu I will need your help integrating the arocc's GenerateDef changes with zig's build.zig - I'm not sure how to add the def files to the aro module.

[Vexu]

I don't think there's going to be an easy way to do so because of the bootstrapping process. I'd just copy the generated files from the cache and replace the def files.

[Vexu]

One possible option would be to add a stubbed def file and then in CMake add that as the module for the def files for the purpose of bootstrapping. For building them regularly it should be easy enough to copy some parts from Aro's build.zig.

This should enable bootstrapping:

pubfnwith(comptimeProperties: type) type {
returnstruct {
tag: Tag=@enumFromInt(0),
properties: Properties=undefined,
pubconstmax_param_count=1;
pubconstlongest_name=0;
pubconstdata= [_]@This(){.{}};
pubinlinefnfromName(_: []constu8) ?@This() {
returnif (@hasField(Properties, "declspec")) nullelse .{};
}
pubfnnameFromUniqueIndex(_: u16, _: []u8) []u8 {
return"";
}
pubfnuniqueIndex(_: []constu8) ?u16 {
returnnull;
}
pubconstTag=enum(u16) { _ };
pubfnnameFromTag(_: Tag) NameBuf {
return .{};
}
pubfntagFromName(name: []constu8) ?Tag {
return@enumFromInt(name.len);
}
pubconstNameBuf=struct {
pubfnspan(_: *constNameBuf) []constu8 {
return"";
}
};
};
}

[Vexu]

I forgot about the "file exists in multiple modules" error again and made the stub work for both. It could be simplified for them individually but this works too.

I simplified the names stub while fixing the build error.

[ehaas]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files. I fixed it with the following diff:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index dcbddc476..d5fd9b4b1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -647,8 +647,11 @@ set(ZIG_STAGE2_SOURCES
"${CMAKE_SOURCE_DIR}/src/windows_sdk.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_builtins.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_attributes.zig"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Attribute/names.def"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Builtins/Builtin.def"
)
+
if(MSVC)
set(MSVC_DIA_SDK_DIR "$ENV{VSINSTALLDIR}DIA SDK")
if(IS_DIRECTORY ${MSVC_DIA_SDK_DIR})
@@ -947,3 +950,10 @@ install(CODE "set(ZIG_BUILD_ARGS \"${ZIG_BUILD_ARGS}\")")
install(CODE "set(CMAKE_INSTALL_PREFIX \"${CMAKE_INSTALL_PREFIX}\")")
install(CODE "set(CMAKE_SOURCE_DIR \"${CMAKE_SOURCE_DIR}\")")
install(SCRIPT "${CMAKE_SOURCE_DIR}/cmake/install.cmake")
+
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Attribute/names.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Attribute)
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Builtins/Builtin.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Builtins)

I'm not sure if this is the correct fix; I also did this without your changes to this branch or to aro from today.

[andrewrk]

Please don't add any additional logic to CMakeLists.txt. Zig should depend on Aro only as a set of zig files. I don't want to add any Aro-related build system logic to zig's build process.

[Vexu]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files.

I think the issue was Vexu/arocc@9f4c28a

[ehaas]

If we want to completely eliminate the CMakeLists.txt changes (so, no stub modules or anything) I have a local patch that does that by replacing the .def file contents with the generated versions. It succeeds at the bootstrap + stage4 build. Let me know if that is desired and I can push it.

[andrewrk]

Thanks @ehaas. I think this is fine - passing a few more args to building zig2.c is OK, I just didn't want to add any more steps to the process. For example, to update #17892 after merging this PR, I will only need to adjust a few command line args, but not introduce more logic into bootstrap.c.

[ehaas]

Based on manual testing, this produces the same output for kernel32.def when compared to clang, other than minor whitespace differences (clang puts a newline at the beginning, and a trailing space on one line which aro does not)

Testing steps:

1. Clear zig cache
2. Create empty test.c
3. With status-quo zig: zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print a zig clang line to stderr; the output path is the final argument
4. With this branch: ./zig-out/bin/zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print output path: <ZIG_CACHE_PATH>/o/<SOME HASH>/kernel32.def
5. Diff the two files
6. Available targets are x86_64-windows-gnu, x86-windows-gnu, arm-windows-gnu, aarch64-windows-gnu

I manually tested error handling by editing kernel32.def.in to have an invalid preprocessing directive and got the following:

zig-out/lib/zig/libc/mingw/lib-common/kernel32.def.in:1:1: error: invalid preprocessing directive
#foo
^
1 error generated.
error: unable to generate DLL import .lib file for kernel32: AroPreprocessorFailed

Closes#17753

[andrewrk]

I suggest to put a check for build_options.only_core_functionality to prevent Aro from being compiled into zig1.wasm or zig2.c during the bootstrap process. This will also fix the current CI failures.

[ehaas]

Is there a way to test this locally if I don't have access to a windows machine? I don't want to spam the CI with attempts, since I'm not totally sure where the check should go. It seems like the function is being used, since we're hitting that panic - does that mean that if build_options.only_core_functionality is true, we should use the existing zig clang solution, and otherwise use the new aro one?

[andrewrk]

Yes, here is a way to test it:

1. put @compileError() somewhere inside aro
2. zig build -Donly-c

If you get the compile error, it's incorrectly including aro in the build.

The check goes at the top of buildImportLib:

if (build_options.only_core_functionality) @panic("building import libs not included in core functionality");

[andrewrk]

Oops, I gave you some bad advice. It should be checking only_c instead of only_core_functionality.

We don't want the logic to be part of zig1.wasm, however we do want the logic to be part of zig2.c.

The use case for only_core_functionality is for example package fetching. We certainly do not need zig2 to have the package manager logic inside of it.

[ehaas]

Ok, if I add that I assume it means it's also safe for me to remove the if (builtin.zig_backend == .stage2_c) @panic("the CBE cannot compile Aro yet!"); from later in the function, right before I import aro?

[andrewrk]

Ah, I forgot about this. I'll open another issue for making zig support writing import libraries from def files.

[andrewrk]

#17807

[ehaas]

@Vexu I will need your help integrating the arocc's GenerateDef changes with zig's build.zig - I'm not sure how to add the def files to the aro module.

[Vexu]

I don't think there's going to be an easy way to do so because of the bootstrapping process. I'd just copy the generated files from the cache and replace the def files.

[Vexu]

One possible option would be to add a stubbed def file and then in CMake add that as the module for the def files for the purpose of bootstrapping. For building them regularly it should be easy enough to copy some parts from Aro's build.zig.

This should enable bootstrapping:

pubfnwith(comptimeProperties: type) type {
returnstruct {
tag: Tag=@enumFromInt(0),
properties: Properties=undefined,
pubconstmax_param_count=1;
pubconstlongest_name=0;
pubconstdata= [_]@This(){.{}};
pubinlinefnfromName(_: []constu8) ?@This() {
returnif (@hasField(Properties, "declspec")) nullelse .{};
}
pubfnnameFromUniqueIndex(_: u16, _: []u8) []u8 {
return"";
}
pubfnuniqueIndex(_: []constu8) ?u16 {
returnnull;
}
pubconstTag=enum(u16) { _ };
pubfnnameFromTag(_: Tag) NameBuf {
return .{};
}
pubfntagFromName(name: []constu8) ?Tag {
return@enumFromInt(name.len);
}
pubconstNameBuf=struct {
pubfnspan(_: *constNameBuf) []constu8 {
return"";
}
};
};
}

[Vexu]

I forgot about the "file exists in multiple modules" error again and made the stub work for both. It could be simplified for them individually but this works too.

I simplified the names stub while fixing the build error.

[ehaas]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files. I fixed it with the following diff:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index dcbddc476..d5fd9b4b1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -647,8 +647,11 @@ set(ZIG_STAGE2_SOURCES
"${CMAKE_SOURCE_DIR}/src/windows_sdk.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_builtins.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_attributes.zig"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Attribute/names.def"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Builtins/Builtin.def"
)
+
if(MSVC)
set(MSVC_DIA_SDK_DIR "$ENV{VSINSTALLDIR}DIA SDK")
if(IS_DIRECTORY ${MSVC_DIA_SDK_DIR})
@@ -947,3 +950,10 @@ install(CODE "set(ZIG_BUILD_ARGS \"${ZIG_BUILD_ARGS}\")")
install(CODE "set(CMAKE_INSTALL_PREFIX \"${CMAKE_INSTALL_PREFIX}\")")
install(CODE "set(CMAKE_SOURCE_DIR \"${CMAKE_SOURCE_DIR}\")")
install(SCRIPT "${CMAKE_SOURCE_DIR}/cmake/install.cmake")
+
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Attribute/names.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Attribute)
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Builtins/Builtin.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Builtins)

I'm not sure if this is the correct fix; I also did this without your changes to this branch or to aro from today.

[andrewrk]

Please don't add any additional logic to CMakeLists.txt. Zig should depend on Aro only as a set of zig files. I don't want to add any Aro-related build system logic to zig's build process.

[Vexu]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files.

I think the issue was Vexu/arocc@9f4c28a

[ehaas]

If we want to completely eliminate the CMakeLists.txt changes (so, no stub modules or anything) I have a local patch that does that by replacing the .def file contents with the generated versions. It succeeds at the bootstrap + stage4 build. Let me know if that is desired and I can push it.

[andrewrk]

Thanks @ehaas. I think this is fine - passing a few more args to building zig2.c is OK, I just didn't want to add any more steps to the process. For example, to update #17892 after merging this PR, I will only need to adjust a few command line args, but not introduce more logic into bootstrap.c.

[ehaas]

Based on manual testing, this produces the same output for kernel32.def when compared to clang, other than minor whitespace differences (clang puts a newline at the beginning, and a trailing space on one line which aro does not)

Testing steps:

1. Clear zig cache
2. Create empty test.c
3. With status-quo zig: zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print a zig clang line to stderr; the output path is the final argument
4. With this branch: ./zig-out/bin/zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print output path: <ZIG_CACHE_PATH>/o/<SOME HASH>/kernel32.def
5. Diff the two files
6. Available targets are x86_64-windows-gnu, x86-windows-gnu, arm-windows-gnu, aarch64-windows-gnu

I manually tested error handling by editing kernel32.def.in to have an invalid preprocessing directive and got the following:

zig-out/lib/zig/libc/mingw/lib-common/kernel32.def.in:1:1: error: invalid preprocessing directive
#foo
^
1 error generated.
error: unable to generate DLL import .lib file for kernel32: AroPreprocessorFailed

Closes#17753

[andrewrk]

I suggest to put a check for build_options.only_core_functionality to prevent Aro from being compiled into zig1.wasm or zig2.c during the bootstrap process. This will also fix the current CI failures.

[ehaas]

Is there a way to test this locally if I don't have access to a windows machine? I don't want to spam the CI with attempts, since I'm not totally sure where the check should go. It seems like the function is being used, since we're hitting that panic - does that mean that if build_options.only_core_functionality is true, we should use the existing zig clang solution, and otherwise use the new aro one?

[andrewrk]

Yes, here is a way to test it:

1. put @compileError() somewhere inside aro
2. zig build -Donly-c

If you get the compile error, it's incorrectly including aro in the build.

The check goes at the top of buildImportLib:

if (build_options.only_core_functionality) @panic("building import libs not included in core functionality");

[andrewrk]

Oops, I gave you some bad advice. It should be checking only_c instead of only_core_functionality.

We don't want the logic to be part of zig1.wasm, however we do want the logic to be part of zig2.c.

The use case for only_core_functionality is for example package fetching. We certainly do not need zig2 to have the package manager logic inside of it.

[ehaas]

Ok, if I add that I assume it means it's also safe for me to remove the if (builtin.zig_backend == .stage2_c) @panic("the CBE cannot compile Aro yet!"); from later in the function, right before I import aro?

[andrewrk]

Ah, I forgot about this. I'll open another issue for making zig support writing import libraries from def files.

[andrewrk]

#17807

[ehaas]

@Vexu I will need your help integrating the arocc's GenerateDef changes with zig's build.zig - I'm not sure how to add the def files to the aro module.

[Vexu]

I don't think there's going to be an easy way to do so because of the bootstrapping process. I'd just copy the generated files from the cache and replace the def files.

[Vexu]

One possible option would be to add a stubbed def file and then in CMake add that as the module for the def files for the purpose of bootstrapping. For building them regularly it should be easy enough to copy some parts from Aro's build.zig.

This should enable bootstrapping:

pubfnwith(comptimeProperties: type) type {
returnstruct {
tag: Tag=@enumFromInt(0),
properties: Properties=undefined,
pubconstmax_param_count=1;
pubconstlongest_name=0;
pubconstdata= [_]@This(){.{}};
pubinlinefnfromName(_: []constu8) ?@This() {
returnif (@hasField(Properties, "declspec")) nullelse .{};
}
pubfnnameFromUniqueIndex(_: u16, _: []u8) []u8 {
return"";
}
pubfnuniqueIndex(_: []constu8) ?u16 {
returnnull;
}
pubconstTag=enum(u16) { _ };
pubfnnameFromTag(_: Tag) NameBuf {
return .{};
}
pubfntagFromName(name: []constu8) ?Tag {
return@enumFromInt(name.len);
}
pubconstNameBuf=struct {
pubfnspan(_: *constNameBuf) []constu8 {
return"";
}
};
};
}

[Vexu]

I forgot about the "file exists in multiple modules" error again and made the stub work for both. It could be simplified for them individually but this works too.

I simplified the names stub while fixing the build error.

[ehaas]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files. I fixed it with the following diff:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index dcbddc476..d5fd9b4b1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -647,8 +647,11 @@ set(ZIG_STAGE2_SOURCES
"${CMAKE_SOURCE_DIR}/src/windows_sdk.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_builtins.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_attributes.zig"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Attribute/names.def"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Builtins/Builtin.def"
)
+
if(MSVC)
set(MSVC_DIA_SDK_DIR "$ENV{VSINSTALLDIR}DIA SDK")
if(IS_DIRECTORY ${MSVC_DIA_SDK_DIR})
@@ -947,3 +950,10 @@ install(CODE "set(ZIG_BUILD_ARGS \"${ZIG_BUILD_ARGS}\")")
install(CODE "set(CMAKE_INSTALL_PREFIX \"${CMAKE_INSTALL_PREFIX}\")")
install(CODE "set(CMAKE_SOURCE_DIR \"${CMAKE_SOURCE_DIR}\")")
install(SCRIPT "${CMAKE_SOURCE_DIR}/cmake/install.cmake")
+
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Attribute/names.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Attribute)
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Builtins/Builtin.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Builtins)

I'm not sure if this is the correct fix; I also did this without your changes to this branch or to aro from today.

[andrewrk]

Please don't add any additional logic to CMakeLists.txt. Zig should depend on Aro only as a set of zig files. I don't want to add any Aro-related build system logic to zig's build process.

[Vexu]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files.

I think the issue was Vexu/arocc@9f4c28a

[ehaas]

If we want to completely eliminate the CMakeLists.txt changes (so, no stub modules or anything) I have a local patch that does that by replacing the .def file contents with the generated versions. It succeeds at the bootstrap + stage4 build. Let me know if that is desired and I can push it.

[andrewrk]

Thanks @ehaas. I think this is fine - passing a few more args to building zig2.c is OK, I just didn't want to add any more steps to the process. For example, to update #17892 after merging this PR, I will only need to adjust a few command line args, but not introduce more logic into bootstrap.c.

[ehaas]

Based on manual testing, this produces the same output for kernel32.def when compared to clang, other than minor whitespace differences (clang puts a newline at the beginning, and a trailing space on one line which aro does not)

Testing steps:

1. Clear zig cache
2. Create empty test.c
3. With status-quo zig: zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print a zig clang line to stderr; the output path is the final argument
4. With this branch: ./zig-out/bin/zig build-lib -target x86_64-windows-gnu -lkernel32 --verbose-cc test.c
   a. This will print output path: <ZIG_CACHE_PATH>/o/<SOME HASH>/kernel32.def
5. Diff the two files
6. Available targets are x86_64-windows-gnu, x86-windows-gnu, arm-windows-gnu, aarch64-windows-gnu

I manually tested error handling by editing kernel32.def.in to have an invalid preprocessing directive and got the following:

zig-out/lib/zig/libc/mingw/lib-common/kernel32.def.in:1:1: error: invalid preprocessing directive
#foo
^
1 error generated.
error: unable to generate DLL import .lib file for kernel32: AroPreprocessorFailed

Closes#17753

[andrewrk]

I suggest to put a check for build_options.only_core_functionality to prevent Aro from being compiled into zig1.wasm or zig2.c during the bootstrap process. This will also fix the current CI failures.

[ehaas]

Is there a way to test this locally if I don't have access to a windows machine? I don't want to spam the CI with attempts, since I'm not totally sure where the check should go. It seems like the function is being used, since we're hitting that panic - does that mean that if build_options.only_core_functionality is true, we should use the existing zig clang solution, and otherwise use the new aro one?

[andrewrk]

Yes, here is a way to test it:

1. put @compileError() somewhere inside aro
2. zig build -Donly-c

If you get the compile error, it's incorrectly including aro in the build.

The check goes at the top of buildImportLib:

if (build_options.only_core_functionality) @panic("building import libs not included in core functionality");

[andrewrk]

Oops, I gave you some bad advice. It should be checking only_c instead of only_core_functionality.

We don't want the logic to be part of zig1.wasm, however we do want the logic to be part of zig2.c.

The use case for only_core_functionality is for example package fetching. We certainly do not need zig2 to have the package manager logic inside of it.

[ehaas]

Ok, if I add that I assume it means it's also safe for me to remove the if (builtin.zig_backend == .stage2_c) @panic("the CBE cannot compile Aro yet!"); from later in the function, right before I import aro?

[andrewrk]

Ah, I forgot about this. I'll open another issue for making zig support writing import libraries from def files.

[andrewrk]

#17807

[ehaas]

@Vexu I will need your help integrating the arocc's GenerateDef changes with zig's build.zig - I'm not sure how to add the def files to the aro module.

[Vexu]

I don't think there's going to be an easy way to do so because of the bootstrapping process. I'd just copy the generated files from the cache and replace the def files.

[Vexu]

One possible option would be to add a stubbed def file and then in CMake add that as the module for the def files for the purpose of bootstrapping. For building them regularly it should be easy enough to copy some parts from Aro's build.zig.

This should enable bootstrapping:

pubfnwith(comptimeProperties: type) type {
returnstruct {
tag: Tag=@enumFromInt(0),
properties: Properties=undefined,
pubconstmax_param_count=1;
pubconstlongest_name=0;
pubconstdata= [_]@This(){.{}};
pubinlinefnfromName(_: []constu8) ?@This() {
returnif (@hasField(Properties, "declspec")) nullelse .{};
}
pubfnnameFromUniqueIndex(_: u16, _: []u8) []u8 {
return"";
}
pubfnuniqueIndex(_: []constu8) ?u16 {
returnnull;
}
pubconstTag=enum(u16) { _ };
pubfnnameFromTag(_: Tag) NameBuf {
return .{};
}
pubfntagFromName(name: []constu8) ?Tag {
return@enumFromInt(name.len);
}
pubconstNameBuf=struct {
pubfnspan(_: *constNameBuf) []constu8 {
return"";
}
};
};
}

[Vexu]

I forgot about the "file exists in multiple modules" error again and made the stub work for both. It could be simplified for them individually but this works too.

I simplified the names stub while fixing the build error.

[ehaas]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files. I fixed it with the following diff:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index dcbddc476..d5fd9b4b1 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -647,8 +647,11 @@ set(ZIG_STAGE2_SOURCES
"${CMAKE_SOURCE_DIR}/src/windows_sdk.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_builtins.zig"
"${CMAKE_SOURCE_DIR}/src/stubs/aro_attributes.zig"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Attribute/names.def"
+ "${CMAKE_SOURCE_DIR}/deps/aro/Builtins/Builtin.def"
)
+
if(MSVC)
set(MSVC_DIA_SDK_DIR "$ENV{VSINSTALLDIR}DIA SDK")
if(IS_DIRECTORY ${MSVC_DIA_SDK_DIR})
@@ -947,3 +950,10 @@ install(CODE "set(ZIG_BUILD_ARGS \"${ZIG_BUILD_ARGS}\")")
install(CODE "set(CMAKE_INSTALL_PREFIX \"${CMAKE_INSTALL_PREFIX}\")")
install(CODE "set(CMAKE_SOURCE_DIR \"${CMAKE_SOURCE_DIR}\")")
install(SCRIPT "${CMAKE_SOURCE_DIR}/cmake/install.cmake")
+
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Attribute/names.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Attribute)
+
+file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/deps/aro/Builtins/Builtin.def
+ DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/deps/aro/Builtins)

I'm not sure if this is the correct fix; I also did this without your changes to this branch or to aro from today.

[andrewrk]

Please don't add any additional logic to CMakeLists.txt. Zig should depend on Aro only as a set of zig files. I don't want to add any Aro-related build system logic to zig's build process.

[Vexu]

The issue I was seeing was that stage3 was unable to build the zig compiler due to not finding the def files.

I think the issue was Vexu/arocc@9f4c28a

[ehaas]

If we want to completely eliminate the CMakeLists.txt changes (so, no stub modules or anything) I have a local patch that does that by replacing the .def file contents with the generated versions. It succeeds at the bootstrap + stage4 build. Let me know if that is desired and I can push it.

[andrewrk]

Thanks @ehaas. I think this is fine - passing a few more args to building zig2.c is OK, I just didn't want to add any more steps to the process. For example, to update #17892 after merging this PR, I will only need to adjust a few command line args, but not introduce more logic into bootstrap.c.