Haiku: Add Haiku support for CoreCLR/PAL by trungnt2910 · Pull Request #93907 · dotnet/runtime

trungnt2910 · 2023-10-24T04:40:55Z

This contains the code required to build this repo's native components and get paltests to pass on Haiku.

Most implementation details related to this commit are documented here.

Part of #55803.

janvorli · 2023-10-24T17:21:07Z

I would prefer the #elif defined(TARGET_LINUX) here and final #else with the posix fallback instead of hierarchical ifdef.

Done. The final #else assigns the fd variable to -1 instead of doing the POSIX fallback itself, in case any of the platform-specific functions above return an error.
This should solve #72192 from the .NET 7 era.

janvorli · 2023-10-24T22:38:19Z

Should we keep the old aborting code as a fallback and add separate #if for HAIKU instead? I am not familiar enough with the contract of this function though.

I've added back the abort() logic, as an assertion that the function is unreachable.
This is true because on unsupported platforms, the higher-level C# libraries should already throw while trying to construct a network events socket.
Returning Error_SUCCESS does not make much sense though.

trungnt2910 · 2023-10-25T07:34:18Z

Strange, I'm not modifying any FreeBSD-related codepaths, what's wrong with the FreeBSD build?

jkotas · 2023-10-25T13:08:55Z

FreeBSD build error is:

/__w/1/s/src/coreclr/pal/src/misc/sysinfo.cpp:215:57: error: use of undeclared identifier 'NUPML4E'
lpSystemInfo->lpMaximumApplicationAddress = (PVOID) VM_MAXUSER_ADDRESS;
^
/crossrootfs/x64/usr/include/machine/vmparam.h:186:35: note: expanded from macro 'VM_MAXUSER_ADDRESS'
#define VM_MAXUSER_ADDRESS UVADDR(NUPML4E, 0, 0, 0)
^

I guess you have introduced it by changing conditions around some of the includes.

trungnt2910 · 2023-10-25T13:41:22Z

This is suspicious:

-- Looking for include file sys/user.h
-- Looking for include file sys/user.h - not found

I wonder how I could get more detailed CMake logs for the CI runs.

trungnt2910 · 2023-10-25T13:55:53Z

When looking at other files, it seems like sys/user.h is only used on FreeBSD. Somehow sysinfo.cpp includes this header for all platforms.

This time, instead of checking for this header during configuration as in #86391, a FreeBSD #ifdef guard is added instead.

huoyaoyuan · 2023-10-30T06:13:58Z

+ // POSIX fallback
+ if (fd == -1)
+ {
+ char name[24];
+ sprintf(name, "/shm-dotnet-%d", getpid());
+ name[sizeof(name) - 1] = '\0';
+ shm_unlink(name);
+ fd = shm_open(name, O_RDWR | O_CREAT | O_EXCL | O_NOFOLLOW, 0600);
+ shm_unlink(name);
+ }


Shouldn't this block be in #else?

No, it should not, as stated in #93907 (comment).
Even for OSes with platform-specific ways to get a special memory file open, should their method fail, we can still always use the POSIX fallback.

huoyaoyuan · 2023-10-30T06:15:16Z

+#ifdef TARGET_LINUX
+#include <sys/syscall.h>
+#endif


Is this used by other Unix systems?

No, it is only used on Linux:
runtime/src/coreclr/gc/unix/numasupport.cpp
Lines 52 to 63 in e5c631d
#ifdef TARGET_LINUX
if (syscall(__NR_get_mempolicy, NULL, NULL, 0, 0, 0) < 0 && errno == ENOSYS)
return;
int highestNumaNode = GetNodeNum("/sys/devices/system/node", false);
// we only use this implementation when there are two or more NUMA nodes available
if (highestNumaNode < 1)
return;
g_numaAvailable = true;
g_highestNumaNode = highestNumaNode;
#endif
and
runtime/src/coreclr/gc/unix/numasupport.cpp
Lines 81 to 83 in e5c631d
#ifdef TARGET_LINUX
returnsyscall(__NR_mbind, (long)start, len, 1, (long)nodemask, maxnode, 0);
#else

huoyaoyuan · 2023-10-30T06:17:55Z

 #endif // !HOST_64BIT
-#endif // !PAL_STDCPP_COMPAT
 #define _SIZE_T_DEFINED
+#endif // !PAL_STDCPP_COMPAT


What's the impact range of this?

What do you mean by "impact range"?
The line should affect all platforms. However, I believe that this is how things should be, since with PAL_STDCPP_COMPAT the header should not define things that potentially mess up system headers.
Also, CoreCLR builds are working, so it should not break the definition on any other platforms.
Some more comments about this change here: #55803 (comment)

trungnt2910 · 2024-04-30T15:57:47Z

Latest Haiku builds are failing with this error:

 [ 7%] Building CXX object inc/CMakeFiles/corguids.dir/__/pal/prebuilt/idl/cordebug_i.cpp.o
In file included from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/palrt.h:630,
from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/rpc.h:15,
from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/prebuilt/idl/cordebug_i.cpp:29:
/home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/safecrt.h:1093:16: error: conflicting declaration of C function 'size_t wcsnlen(const WCHAR*, size_t)'
1093 | size_t __cdecl wcsnlen(const WCHAR *inString, size_t inMaxSize);
| ^~~~~~~
In file included from /home/runner/work/dotnet-builds/dotnet-builds/dotnet-rootfs/boot/system/develop/headers/posix/wctype.h:10,
from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/pal.h:51,
from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/palrt.h:136:
/home/runner/work/dotnet-builds/dotnet-builds/dotnet-rootfs/boot/system/develop/headers/posix/wchar.h:122:17: note: previous declaration 'size_t wcsnlen(const wchar_t*, size_t)'
122 | extern size_t wcsnlen(const wchar_t *wcs, size_t maxLength);
| ^~~~~~~
/home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/safecrt.h:1098:16: error: conflicting declaration of C function 'size_t wcsnlen(const WCHAR*, size_t)'
1098 | size_t __cdecl wcsnlen(const WCHAR *inString, size_t inMaxSize)
| ^~~~~~~
/home/runner/work/dotnet-builds/dotnet-builds/dotnet-rootfs/boot/system/develop/headers/posix/wchar.h:122:17: note: previous declaration 'size_t wcsnlen(const wchar_t*, size_t)'
122 | extern size_t wcsnlen(const wchar_t *wcs, size_t maxLength);
| ^~~~~~~
make[2]: *** [inc/CMakeFiles/corguids.dir/build.make:76: inc/CMakeFiles/corguids.dir/__/pal/prebuilt/idl/cordebug_i.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:3250: inc/CMakeFiles/corguids.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Seems like whoever included wctype.h did not expect wchar.h to be included as well.

According to the POSIX Standard, however:

[CX] [Option Start] Inclusion of the <wctype.h> header may make visible all symbols from the headers <ctype.h>, <stdarg.h>, <stddef.h>, <stdio.h>, <stdlib.h>, <string.h>, <time.h>, and <wchar.h>. [Option End]

What should I do in this case? Removing <wchar.h> from Haiku's system include just breaks everything. And according to POSIX Haiku isn't doing anything wrong in the first place...

jkotas · 2024-04-30T16:18:35Z

Does anything break if our local wcsnlen implementation is deleted?

trungnt2910 · 2024-04-30T17:02:37Z

Hasn't broken my local Linux build and moved my local Haiku build to a different error. Let the CI inform the rest of the answer.

trungnt2910 · 2024-04-30T17:24:46Z

The next Haiku error is:

 asmparse.y:176: error: "_IMPORT" redefined [-Werror]
In file included from /home/trung/dotnet-rootfs/boot/system/develop/headers/posix/sys/types.h:11,
from /home/trung/dotnet-rootfs/boot/system/develop/headers/posix/stdio.h:9,
from /home/trung/dotnet-rootfs/boot/system/develop/headers/bsd/stdio.h:9,
from /home/trung/dotnet-runtime/src/coreclr/pal/inc/pal.h:39,
from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/palrt.h:136,
from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/rpc.h:15,
from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/objidl.h:12,
from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/ole2.h:8,
from /home/trung/dotnet-runtime/src/coreclr/inc/cor.h:16,
from /home/trung/dotnet-runtime/src/coreclr/ilasm/./ilasmpch.h:8,
from asmparse.y:9:
/home/trung/dotnet-rootfs/boot/system/develop/headers/os/BeBuild.h:95: note: this is the location of the previous definition
95 | #define _IMPORT
|
asmparse.y:262: error: "_EXPORT" redefined [-Werror]
/home/trung/dotnet-rootfs/boot/system/develop/headers/os/BeBuild.h:90: note: this is the location of the previous definition
90 | # define _EXPORT __attribute__((visibility("default")))
|

Currently I am solving this by adding these lines to ilasmpch.h:

#include "mdfileformat.h"#include "stgpooli.h"+#ifdef _EXPORT+#undef _EXPORT+#endif+#ifdef _IMPORT+#undef _IMPORT+#endif#endif

I wonder whether these lines should be put directly to pal.h instead.

mangod9 · 2024-07-29T16:19:05Z

Hey @trungnt2910, what is the status of this PR. Are you continuing to work on it?

trungnt2910 · 2024-07-29T20:50:31Z

There's still an unanswered question from me above.

When we have a solution for those macros, I can proceed to resolving the conflicts and the PR is ready for further review.

mangod9 · 2024-07-30T19:59:05Z

ok, so the question is about whether to add the undefs in pal.h? We are also triaging PRs for .NET 9, assume this can wait a few weeks ?

trungnt2910 · 2024-07-30T23:07:25Z

Yes, the question is related to where I should add the #undefs, or otherwise how I should fix the redefined error.
I believe that one of the uses of pal.h is to isolate parts of CoreCLR code from system headers, so I initially thought putting those macros there would work.

assume this can wait a few weeks ?

Yeah. This has been waiting for quite a few months now, waiting a few more weeks wouldn't hurt.

mangod9 · 2024-09-16T14:44:19Z

adding @janvorli on advise on #undefs in pal.h.

janvorli · 2024-11-05T18:08:20Z

I wonder whether these lines should be put directly to pal.h instead.

This is strictly ilasm specific, so let's keep it where you have put it.

janvorli

Besides the two comments, it all looks good.

This contains a part of the code required to build CoreCLR and get `paltests` to pass on Haiku. This commit covers `src/coreclr/pal/**`. Co-authored-by: Jessica Hamilton <jessica.l.hamilton@gmail.com>

Some OSes like Haiku define the `_EXPORT` and `_IMPORT` marcos. Symbols beginning with an underscore followed immediately by an uppercase letter are reserved anyway. Add the guards to get them out of the way.

This flag is a compatibility flag and not required on most OSes. Some, including Haiku, does not even define it.

janvorli

LGTM, thank you!

janvorli · 2024-11-05T22:46:38Z

/azp run runtime-coreclr outerloop

azure-pipelines · 2024-11-05T22:46:58Z

Azure Pipelines successfully started running 1 pipeline(s).

janvorli · 2024-11-06T10:48:18Z

The outerloop failure is a known crossgen2 issue #104340

trungnt2910 · 2024-11-06T12:44:40Z

Thanks for the review!

Following up with #109580, hope you could have a look at that.

trungnt2910 requested review from MichalStrehovsky, lambdageek and vargaz as code owners October 24, 2023 04:40

ghost added community-contribution Indicates that the PR has been added by a community member area-PAL-coreclr only for closed issues labels Oct 24, 2023

jkotas reviewed Oct 24, 2023
View reviewed changes

Comment threadsrc/native/minipal/errno.h Outdated

jkotas requested a review from janvorli October 24, 2023 04:55

This was referenced Oct 24, 2023
Tracking issue for CI build timeouts #76454
Closed
Test failure: Wasm.Build.Tests.NativeBuildTests.MonoAOTCross_WorksWithNoTrimming #93522
Closed

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from f98a038 to 3ff7e01Compare October 24, 2023 21:58

janvorli reviewed Oct 24, 2023
View reviewed changes

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch 2 times, most recently from 790029d to 2131127Compare October 25, 2023 04:03

This was referenced Oct 25, 2023
Test_EventSource_EtwManifestGeneration* tests failing in CI #48798
Closed
[wasm][debugger] Debugger tests disabled #89409
Open
The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451
Closed

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from ed1af42 to 4a8cc01Compare October 25, 2023 13:53

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch 4 times, most recently from 0a97fb8 to f1f4369Compare October 28, 2023 20:58

This was referenced Oct 28, 2023
Timeout in System.Net.Quic.Functional.Tests #86019
Closed
CI error: System.Net.Quic.QuicException: The connection timed out from inactivity #91757
Closed
System.Security.Cryptography.Tests timing out #93840
Closed

huoyaoyuan reviewed Oct 30, 2023
View reviewed changes

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from f1f4369 to 72e7cc3Compare November 5, 2023 00:40

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from a7afb88 to c11ed1aCompare April 30, 2024 14:46

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from ed24de7 to 56ff51fCompare April 30, 2024 17:02

build-analysisBot mentioned this pull request Apr 30, 2024
System.Numerics.Tensors.Tests.SingleGenericTensorPrimitives.SpanScalarDestination_SpecialValues fails #101721
Closed

janvorli reviewed Nov 5, 2024
View reviewed changes

Comment threadsrc/coreclr/pal/src/include/pal/thread.hpp Outdated
Comment threadsrc/coreclr/pal/src/map/map.cpp Outdated

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from 56ff51f to 7e63089Compare November 5, 2024 22:34

janvorli reviewed Nov 5, 2024
View reviewed changes

Comment threadsrc/native/minipal/debugger.c Outdated

trungnt2910and others added 3 commits November 6, 2024 09:39

Haiku: Add Haiku support for CoreCLR/PAL
850ab1a
This contains a part of the code required to build CoreCLR and get `paltests` to pass on Haiku. This commit covers `src/coreclr/pal/**`. Co-authored-by: Jessica Hamilton <jessica.l.hamilton@gmail.com>

ilasm: Haiku build fixes
84a1502
Some OSes like Haiku define the `_EXPORT` and `_IMPORT` marcos. Symbols beginning with an underscore followed immediately by an uppercase letter are reserved anyway. Add the guards to get them out of the way.

pal: Remove usage of MAP_FILE
40b806a
This flag is a compatibility flag and not required on most OSes. Some, including Haiku, does not even define it.

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from 7e63089 to 40b806aCompare November 5, 2024 22:40

janvorli approved these changes Nov 5, 2024
View reviewed changes

This was referenced Nov 6, 2024
chrome-DebuggerTests timing out #108078
Closed
[debugger] Inspector exceptions in debugger tests #109025
Open

janvorli merged commit ffcef3d into dotnet:mainNov 6, 2024

am11 mentioned this pull request Nov 6, 2024
Haiku: add platform support #55803
Open

github-actionsBot locked and limited conversation to collaborators Dec 7, 2024

	#ifdef TARGET_LINUX
	if (syscall(__NR_get_mempolicy, NULL, NULL, 0, 0, 0) < 0 && errno == ENOSYS)
	return;

	int highestNumaNode = GetNodeNum("/sys/devices/system/node", false);
	// we only use this implementation when there are two or more NUMA nodes available
	if (highestNumaNode < 1)
	return;

	g_numaAvailable = true;
	g_highestNumaNode = highestNumaNode;
	#endif

	#ifdef TARGET_LINUX
	returnsyscall(__NR_mbind, (long)start, len, 1, (long)nodemask, maxnode, 0);
	#else

Uh oh!

Conversation

trungnt2910 commented Oct 24, 2023

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

trungnt2910 commented Oct 25, 2023

Uh oh!

jkotas commented Oct 25, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trungnt2910 commented Oct 25, 2023

Uh oh!

trungnt2910 commented Oct 25, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

trungnt2910Oct 30, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

trungnt2910 commented Apr 30, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jkotas commented Apr 30, 2024

Uh oh!

trungnt2910 commented Apr 30, 2024

Uh oh!

trungnt2910 commented Apr 30, 2024

Uh oh!

mangod9 commented Jul 29, 2024

Uh oh!

trungnt2910 commented Jul 29, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mangod9 commented Jul 30, 2024

Uh oh!

trungnt2910 commented Jul 30, 2024

Uh oh!

mangod9 commented Sep 16, 2024

Uh oh!

janvorli commented Nov 5, 2024

Uh oh!

janvorli left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

janvorli left a comment

Choose a reason for hiding this comment

Uh oh!

janvorli commented Nov 5, 2024

Uh oh!

azure-pipelinesBot commented Nov 5, 2024

Uh oh!

janvorli commented Nov 6, 2024

Uh oh!

trungnt2910 commented Nov 6, 2024

Uh oh!

Reviewers

jkotas commented Oct 25, 2023•
edited
Loading

trungnt2910Oct 30, 2023•
edited
Loading

trungnt2910 commented Apr 30, 2024•
edited
Loading

trungnt2910 commented Jul 29, 2024•
edited
Loading