[native] Runtime linking at app build time by grendello · Pull Request #9006 · dotnet/android

grendello · 2024-06-05T19:47:30Z

No description provided.

Context: #9006 This commit prepares ground for dynamic native runtime linking at application build time by moving code around and making it more modular, thus allowing the on-build linking of the runtime to work properly. In particular, this applies to the `p/invoke` handling code which is going to be different between the pre-linked (and shipped) runtime and the one linked on demand during application build. The difference lies in the fact that on-demand build will scan the application for p/invoke usage (after trimming) and will only include functions that are actually called from the managed land. Results of the scan will be used to generate LLVM IR code at build time that references the required functions, which will allow the native linker to remove the remaining ones. Changes implemented here can be committed to `main` and, possibly, released with .NET9 without changing how things currently work.

Otherwise `<FileWrites>` in `_CompileNativeAssemblySources` won't actually get written to the cache and the `GenerateJavaStubsAndAssembly` test fails.

They might contain nested types with interesting methods. Doh.

Unconditionally disable dynamic linking in the AOT profiling test

Testing just a single configuration, it's enough.

In dotnet/android#9006 we are working on linking the .NET for Android runtime dynamically at the application build time. Linking involves all the BCL native libraries, including `System.Security.Cryptography.Native.Android` and one of the goals is to hide all the exported symbols used as p/invokes by the managed BCL libraries. This is because p/invoke calls are handled internally and, with dynamic linking of the runtime, there is no longer any reason to use `dlopen` and `dlsym` to look them up, they are all resolved internally, at the link time. Symbol hiding works fine thanks to the `--exclude-libs` `clang` flag, which makes all the exported symbols in the indicated `.a` archives to not be exported by the linker. However, `System.Security.Cryptography.Native.Android` is special in the sense that it contains one symbol which must not be hidden, `Java_net_dot_android_crypto_DotnetProxyTrustManager_verifyRemoteCertificate`. The above function is a Java `native` method implementation, and it requires that not only its name follows the Java JNI naming rules, but that is also available for the JVM to look up using `dlsym`. I tried using the `--export-dynamic-symbol` clang flag to export **just** this function, but it doesn't appear to work no matter where I put the flag in relation to reference to the `.a` archives. Instead, the problem can be dealt with by putting the JNI function in a separate static library, so that I can link it without changing symbol visibility, while making all the `System.Security.Cryptography.Native.Android` symbols invisible.

grendello · 2024-06-05T19:47:30Z

No description provided.

Context: #9006 This commit prepares ground for dynamic native runtime linking at application build time by moving code around and making it more modular, thus allowing the on-build linking of the runtime to work properly. In particular, this applies to the `p/invoke` handling code which is going to be different between the pre-linked (and shipped) runtime and the one linked on demand during application build. The difference lies in the fact that on-demand build will scan the application for p/invoke usage (after trimming) and will only include functions that are actually called from the managed land. Results of the scan will be used to generate LLVM IR code at build time that references the required functions, which will allow the native linker to remove the remaining ones. Changes implemented here can be committed to `main` and, possibly, released with .NET9 without changing how things currently work.

Otherwise `<FileWrites>` in `_CompileNativeAssemblySources` won't actually get written to the cache and the `GenerateJavaStubsAndAssembly` test fails.

They might contain nested types with interesting methods. Doh.

Unconditionally disable dynamic linking in the AOT profiling test

Testing just a single configuration, it's enough.

In dotnet/android#9006 we are working on linking the .NET for Android runtime dynamically at the application build time. Linking involves all the BCL native libraries, including `System.Security.Cryptography.Native.Android` and one of the goals is to hide all the exported symbols used as p/invokes by the managed BCL libraries. This is because p/invoke calls are handled internally and, with dynamic linking of the runtime, there is no longer any reason to use `dlopen` and `dlsym` to look them up, they are all resolved internally, at the link time. Symbol hiding works fine thanks to the `--exclude-libs` `clang` flag, which makes all the exported symbols in the indicated `.a` archives to not be exported by the linker. However, `System.Security.Cryptography.Native.Android` is special in the sense that it contains one symbol which must not be hidden, `Java_net_dot_android_crypto_DotnetProxyTrustManager_verifyRemoteCertificate`. The above function is a Java `native` method implementation, and it requires that not only its name follows the Java JNI naming rules, but that is also available for the JVM to look up using `dlsym`. I tried using the `--export-dynamic-symbol` clang flag to export **just** this function, but it doesn't appear to work no matter where I put the flag in relation to reference to the `.a` archives. Instead, the problem can be dealt with by putting the JNI function in a separate static library, so that I can link it without changing symbol visibility, while making all the `System.Security.Cryptography.Native.Android` symbols invisible.

grendello · 2024-06-05T19:47:30Z

No description provided.

Context: #9006 This commit prepares ground for dynamic native runtime linking at application build time by moving code around and making it more modular, thus allowing the on-build linking of the runtime to work properly. In particular, this applies to the `p/invoke` handling code which is going to be different between the pre-linked (and shipped) runtime and the one linked on demand during application build. The difference lies in the fact that on-demand build will scan the application for p/invoke usage (after trimming) and will only include functions that are actually called from the managed land. Results of the scan will be used to generate LLVM IR code at build time that references the required functions, which will allow the native linker to remove the remaining ones. Changes implemented here can be committed to `main` and, possibly, released with .NET9 without changing how things currently work.

Otherwise `<FileWrites>` in `_CompileNativeAssemblySources` won't actually get written to the cache and the `GenerateJavaStubsAndAssembly` test fails.

They might contain nested types with interesting methods. Doh.

Unconditionally disable dynamic linking in the AOT profiling test

Testing just a single configuration, it's enough.

In dotnet/android#9006 we are working on linking the .NET for Android runtime dynamically at the application build time. Linking involves all the BCL native libraries, including `System.Security.Cryptography.Native.Android` and one of the goals is to hide all the exported symbols used as p/invokes by the managed BCL libraries. This is because p/invoke calls are handled internally and, with dynamic linking of the runtime, there is no longer any reason to use `dlopen` and `dlsym` to look them up, they are all resolved internally, at the link time. Symbol hiding works fine thanks to the `--exclude-libs` `clang` flag, which makes all the exported symbols in the indicated `.a` archives to not be exported by the linker. However, `System.Security.Cryptography.Native.Android` is special in the sense that it contains one symbol which must not be hidden, `Java_net_dot_android_crypto_DotnetProxyTrustManager_verifyRemoteCertificate`. The above function is a Java `native` method implementation, and it requires that not only its name follows the Java JNI naming rules, but that is also available for the JVM to look up using `dlsym`. I tried using the `--export-dynamic-symbol` clang flag to export **just** this function, but it doesn't appear to work no matter where I put the flag in relation to reference to the `.a` archives. Instead, the problem can be dealt with by putting the JNI function in a separate static library, so that I can link it without changing symbol visibility, while making all the `System.Security.Cryptography.Native.Android` symbols invisible.

grendello · 2024-06-05T19:47:30Z

No description provided.

Context: #9006 This commit prepares ground for dynamic native runtime linking at application build time by moving code around and making it more modular, thus allowing the on-build linking of the runtime to work properly. In particular, this applies to the `p/invoke` handling code which is going to be different between the pre-linked (and shipped) runtime and the one linked on demand during application build. The difference lies in the fact that on-demand build will scan the application for p/invoke usage (after trimming) and will only include functions that are actually called from the managed land. Results of the scan will be used to generate LLVM IR code at build time that references the required functions, which will allow the native linker to remove the remaining ones. Changes implemented here can be committed to `main` and, possibly, released with .NET9 without changing how things currently work.

Otherwise `<FileWrites>` in `_CompileNativeAssemblySources` won't actually get written to the cache and the `GenerateJavaStubsAndAssembly` test fails.

They might contain nested types with interesting methods. Doh.

Unconditionally disable dynamic linking in the AOT profiling test

Testing just a single configuration, it's enough.

In dotnet/android#9006 we are working on linking the .NET for Android runtime dynamically at the application build time. Linking involves all the BCL native libraries, including `System.Security.Cryptography.Native.Android` and one of the goals is to hide all the exported symbols used as p/invokes by the managed BCL libraries. This is because p/invoke calls are handled internally and, with dynamic linking of the runtime, there is no longer any reason to use `dlopen` and `dlsym` to look them up, they are all resolved internally, at the link time. Symbol hiding works fine thanks to the `--exclude-libs` `clang` flag, which makes all the exported symbols in the indicated `.a` archives to not be exported by the linker. However, `System.Security.Cryptography.Native.Android` is special in the sense that it contains one symbol which must not be hidden, `Java_net_dot_android_crypto_DotnetProxyTrustManager_verifyRemoteCertificate`. The above function is a Java `native` method implementation, and it requires that not only its name follows the Java JNI naming rules, but that is also available for the JVM to look up using `dlsym`. I tried using the `--export-dynamic-symbol` clang flag to export **just** this function, but it doesn't appear to work no matter where I put the flag in relation to reference to the `.a` archives. Instead, the problem can be dealt with by putting the JNI function in a separate static library, so that I can link it without changing symbol visibility, while making all the `System.Security.Cryptography.Native.Android` symbols invisible.

grendello · 2024-06-05T19:47:30Z

No description provided.

Context: #9006 This commit prepares ground for dynamic native runtime linking at application build time by moving code around and making it more modular, thus allowing the on-build linking of the runtime to work properly. In particular, this applies to the `p/invoke` handling code which is going to be different between the pre-linked (and shipped) runtime and the one linked on demand during application build. The difference lies in the fact that on-demand build will scan the application for p/invoke usage (after trimming) and will only include functions that are actually called from the managed land. Results of the scan will be used to generate LLVM IR code at build time that references the required functions, which will allow the native linker to remove the remaining ones. Changes implemented here can be committed to `main` and, possibly, released with .NET9 without changing how things currently work.

Otherwise `<FileWrites>` in `_CompileNativeAssemblySources` won't actually get written to the cache and the `GenerateJavaStubsAndAssembly` test fails.

They might contain nested types with interesting methods. Doh.

Unconditionally disable dynamic linking in the AOT profiling test

Testing just a single configuration, it's enough.

In dotnet/android#9006 we are working on linking the .NET for Android runtime dynamically at the application build time. Linking involves all the BCL native libraries, including `System.Security.Cryptography.Native.Android` and one of the goals is to hide all the exported symbols used as p/invokes by the managed BCL libraries. This is because p/invoke calls are handled internally and, with dynamic linking of the runtime, there is no longer any reason to use `dlopen` and `dlsym` to look them up, they are all resolved internally, at the link time. Symbol hiding works fine thanks to the `--exclude-libs` `clang` flag, which makes all the exported symbols in the indicated `.a` archives to not be exported by the linker. However, `System.Security.Cryptography.Native.Android` is special in the sense that it contains one symbol which must not be hidden, `Java_net_dot_android_crypto_DotnetProxyTrustManager_verifyRemoteCertificate`. The above function is a Java `native` method implementation, and it requires that not only its name follows the Java JNI naming rules, but that is also available for the JVM to look up using `dlsym`. I tried using the `--export-dynamic-symbol` clang flag to export **just** this function, but it doesn't appear to work no matter where I put the flag in relation to reference to the `.a` archives. Instead, the problem can be dealt with by putting the JNI function in a separate static library, so that I can link it without changing symbol visibility, while making all the `System.Security.Cryptography.Native.Android` symbols invisible.

grendello · 2024-06-05T19:47:30Z

No description provided.

Context: #9006 This commit prepares ground for dynamic native runtime linking at application build time by moving code around and making it more modular, thus allowing the on-build linking of the runtime to work properly. In particular, this applies to the `p/invoke` handling code which is going to be different between the pre-linked (and shipped) runtime and the one linked on demand during application build. The difference lies in the fact that on-demand build will scan the application for p/invoke usage (after trimming) and will only include functions that are actually called from the managed land. Results of the scan will be used to generate LLVM IR code at build time that references the required functions, which will allow the native linker to remove the remaining ones. Changes implemented here can be committed to `main` and, possibly, released with .NET9 without changing how things currently work.

Otherwise `<FileWrites>` in `_CompileNativeAssemblySources` won't actually get written to the cache and the `GenerateJavaStubsAndAssembly` test fails.

They might contain nested types with interesting methods. Doh.

Unconditionally disable dynamic linking in the AOT profiling test

Testing just a single configuration, it's enough.

In dotnet/android#9006 we are working on linking the .NET for Android runtime dynamically at the application build time. Linking involves all the BCL native libraries, including `System.Security.Cryptography.Native.Android` and one of the goals is to hide all the exported symbols used as p/invokes by the managed BCL libraries. This is because p/invoke calls are handled internally and, with dynamic linking of the runtime, there is no longer any reason to use `dlopen` and `dlsym` to look them up, they are all resolved internally, at the link time. Symbol hiding works fine thanks to the `--exclude-libs` `clang` flag, which makes all the exported symbols in the indicated `.a` archives to not be exported by the linker. However, `System.Security.Cryptography.Native.Android` is special in the sense that it contains one symbol which must not be hidden, `Java_net_dot_android_crypto_DotnetProxyTrustManager_verifyRemoteCertificate`. The above function is a Java `native` method implementation, and it requires that not only its name follows the Java JNI naming rules, but that is also available for the JVM to look up using `dlsym`. I tried using the `--export-dynamic-symbol` clang flag to export **just** this function, but it doesn't appear to work no matter where I put the flag in relation to reference to the `.a` archives. Instead, the problem can be dealt with by putting the JNI function in a separate static library, so that I can link it without changing symbol visibility, while making all the `System.Security.Cryptography.Native.Android` symbols invisible.

grendello · 2024-06-05T19:47:30Z

No description provided.

Context: #9006 This commit prepares ground for dynamic native runtime linking at application build time by moving code around and making it more modular, thus allowing the on-build linking of the runtime to work properly. In particular, this applies to the `p/invoke` handling code which is going to be different between the pre-linked (and shipped) runtime and the one linked on demand during application build. The difference lies in the fact that on-demand build will scan the application for p/invoke usage (after trimming) and will only include functions that are actually called from the managed land. Results of the scan will be used to generate LLVM IR code at build time that references the required functions, which will allow the native linker to remove the remaining ones. Changes implemented here can be committed to `main` and, possibly, released with .NET9 without changing how things currently work.

Otherwise `<FileWrites>` in `_CompileNativeAssemblySources` won't actually get written to the cache and the `GenerateJavaStubsAndAssembly` test fails.

They might contain nested types with interesting methods. Doh.

Unconditionally disable dynamic linking in the AOT profiling test

Testing just a single configuration, it's enough.

In dotnet/android#9006 we are working on linking the .NET for Android runtime dynamically at the application build time. Linking involves all the BCL native libraries, including `System.Security.Cryptography.Native.Android` and one of the goals is to hide all the exported symbols used as p/invokes by the managed BCL libraries. This is because p/invoke calls are handled internally and, with dynamic linking of the runtime, there is no longer any reason to use `dlopen` and `dlsym` to look them up, they are all resolved internally, at the link time. Symbol hiding works fine thanks to the `--exclude-libs` `clang` flag, which makes all the exported symbols in the indicated `.a` archives to not be exported by the linker. However, `System.Security.Cryptography.Native.Android` is special in the sense that it contains one symbol which must not be hidden, `Java_net_dot_android_crypto_DotnetProxyTrustManager_verifyRemoteCertificate`. The above function is a Java `native` method implementation, and it requires that not only its name follows the Java JNI naming rules, but that is also available for the JVM to look up using `dlsym`. I tried using the `--export-dynamic-symbol` clang flag to export **just** this function, but it doesn't appear to work no matter where I put the flag in relation to reference to the `.a` archives. Instead, the problem can be dealt with by putting the JNI function in a separate static library, so that I can link it without changing symbol visibility, while making all the `System.Security.Cryptography.Native.Android` symbols invisible.

grendello · 2024-06-05T19:47:30Z

No description provided.

Context: #9006 This commit prepares ground for dynamic native runtime linking at application build time by moving code around and making it more modular, thus allowing the on-build linking of the runtime to work properly. In particular, this applies to the `p/invoke` handling code which is going to be different between the pre-linked (and shipped) runtime and the one linked on demand during application build. The difference lies in the fact that on-demand build will scan the application for p/invoke usage (after trimming) and will only include functions that are actually called from the managed land. Results of the scan will be used to generate LLVM IR code at build time that references the required functions, which will allow the native linker to remove the remaining ones. Changes implemented here can be committed to `main` and, possibly, released with .NET9 without changing how things currently work.

Otherwise `<FileWrites>` in `_CompileNativeAssemblySources` won't actually get written to the cache and the `GenerateJavaStubsAndAssembly` test fails.

They might contain nested types with interesting methods. Doh.

Unconditionally disable dynamic linking in the AOT profiling test

Testing just a single configuration, it's enough.

In dotnet/android#9006 we are working on linking the .NET for Android runtime dynamically at the application build time. Linking involves all the BCL native libraries, including `System.Security.Cryptography.Native.Android` and one of the goals is to hide all the exported symbols used as p/invokes by the managed BCL libraries. This is because p/invoke calls are handled internally and, with dynamic linking of the runtime, there is no longer any reason to use `dlopen` and `dlsym` to look them up, they are all resolved internally, at the link time. Symbol hiding works fine thanks to the `--exclude-libs` `clang` flag, which makes all the exported symbols in the indicated `.a` archives to not be exported by the linker. However, `System.Security.Cryptography.Native.Android` is special in the sense that it contains one symbol which must not be hidden, `Java_net_dot_android_crypto_DotnetProxyTrustManager_verifyRemoteCertificate`. The above function is a Java `native` method implementation, and it requires that not only its name follows the Java JNI naming rules, but that is also available for the JVM to look up using `dlsym`. I tried using the `--export-dynamic-symbol` clang flag to export **just** this function, but it doesn't appear to work no matter where I put the flag in relation to reference to the `.a` archives. Instead, the problem can be dealt with by putting the JNI function in a separate static library, so that I can link it without changing symbol visibility, while making all the `System.Security.Cryptography.Native.Android` symbols invisible.

grendello force-pushed the dev/grendel/runtime-link-at-build branch 4 times, most recently from bf5c35a to bfedf30Compare June 12, 2024 19:08

grendello force-pushed the dev/grendel/runtime-link-at-build branch 4 times, most recently from 9a604d3 to 9fc5594Compare June 27, 2024 19:41

grendello force-pushed the dev/grendel/runtime-link-at-build branch 3 times, most recently from 7f5c98e to 57ab585Compare July 5, 2024 16:06

grendello force-pushed the dev/grendel/runtime-link-at-build branch 4 times, most recently from 58f17e2 to 6f5f5c6Compare July 16, 2024 07:12

grendello mentioned this pull request Jul 16, 2024
Native runtime reorganization #9113
Merged

grendello force-pushed the dev/grendel/runtime-link-at-build branch 2 times, most recently from 68ab1b7 to da87011Compare July 23, 2024 16:24

grendello force-pushed the dev/grendel/runtime-link-at-build branch 6 times, most recently from 5f35bf1 to 99baef4Compare July 31, 2024 08:24

grendello force-pushed the dev/grendel/runtime-link-at-build branch from 99baef4 to 6d17643Compare August 19, 2024 09:00

grendello force-pushed the dev/grendel/runtime-link-at-build branch 3 times, most recently from 9ed6397 to 4d17d92Compare August 29, 2024 19:14

grendello force-pushed the dev/grendel/runtime-link-at-build branch from 92da875 to 813c444Compare September 3, 2024 19:43

grendello added 19 commits November 25, 2024 16:34

Tweak some tests to run with the unified runtime
3daf433

Instant run is no more
2f38764

Try to fix some tests
6c1c742

_LinkNativeRuntime must be called earlier
a0c671f
Otherwise `<FileWrites>` in `_CompileNativeAssemblySources` won't actually get written to the cache and the `GenerateJavaStubsAndAssembly` test fails.

Post-rebase fixups
6f06c21

Don't ignore method-less types when scanning
71749a9
They might contain nested types with interesting methods. Doh.

Let's see if this works
6776200

Mono.Android-Tests might work now
dd9fbba

This might fix the GenerateJavaStubsAndAssembly test
0025d34

AOT profiling won't work with dynamic runtime linking
4dee599
Unconditionally disable dynamic linking in the AOT profiling test

Fix some post-rebase errors
d216289

Don't package BCL libraries when linking dynamically
cb678cf

Add some debug messages
f25989f

Accept all the libs when not linking
fd0406e

Make runtime linking an option in the DotNetBuild test
130514d
Testing just a single configuration, it's enough.

Don't test dynamic linking in here
dcf6fb5

Not needed anymore
3bd5c59

Post-rebase fixes
327687c

Still needed, after all
81a7369

grendello force-pushed the dev/grendel/runtime-link-at-build branch from 7503400 to 81a7369Compare November 25, 2024 15:34

jonathanpeppers removed this from the .NET 10 milestone Aug 29, 2025

grendello closed this Feb 6, 2026

grendello deleted the dev/grendel/runtime-link-at-build branch February 6, 2026 08:47

github-actionsBot locked and limited conversation to collaborators Mar 8, 2026