linux_android_with_fallback: avoid dlsym

[tamird]

See the commit message.

Fixes#600.

[tamird]

r? @newpavlov

[tamird]

@newpavlov I've added a test now to check that musl works when /dev/urandom isn't available.

[newpavlov]

I don't think it's a good approach to solving this issue. We probably should write a MUSL-specific version of the init function which links statically to libc::getrandom and checks at runtime whether it's supported.

As for getrandom_test_linux_fallback_no_dev_urandom, we probably do not need it. If neither libc::getrandom, nor /dev/urandom work, then we simply can not do anything.

[tamird]

I don't think it's a good approach to solving this issue. We probably should write a MUSL-specific version of the init function which links statically to libc::getrandom and checks at runtime whether it's supported.

That's exactly what this does.

As for getrandom_test_linux_fallback_no_dev_urandom, we probably do not need it. If neither libc::getrandom, nor /dev/urandom work, then we simply can not do anything.

getrandom_test_linux_fallback_no_dev_urandom exercises the case where libc::getrandomdoes work but /dev/urandom does not. There is currently no coverage for this case, which is why this bug exists.

[tamird]

@newpavlov thoughts?

[newpavlov]

I created #602 as a smaller alternative.

As for testing, I will do it in a different PR slightly differently.

[tamird]

I've backed out the test.

I've changed the approach here - now the check for getrandom is done entirely at compile time by attempting to link a trivial program that calls getrandom.

The build script needs a bit of refinement - the check is not necessary on all platforms, but I want to get CI results and also to let you have a look.

[newpavlov]

Trying to detect getrandom availability in a build script not only complex, but also a bad approach in general IMO. A binary can be easily built on a machine with getrandom present, but executed on a machine without it or users may cross-compile.

[tamird]

A binary can be easily built on a machine with getrandom present, but executed on a machine without it

The only target we handle that may or may not have getrandom is android, and there we are always cross-compiling using the NDK where the target API level is specified. So I think we do not need to handle the case where getrandom is available at build time but not present in the deployment target.

or users may cross-compile.

Cross-compilation already works correctly with this approach. We are using the user's native compiler for the target.

[newpavlov]

Fixed in #602

[tamird]

Please reopen this. I'm going to rebase it - it is a separate proposal to the fix in #602. This is about removing dlsym entirely.

[newpavlov]

I would prefer to keep using dlsym on non-MUSL targets. Alternatively, we could use raw syscalls similarly to std, but then we would lose on potential vDSO optimizations.

[tamird]

Let's continue the conversation over code, please. I'd appreciate some reasoning. Why is dlsym preferable?

[newpavlov]

It's much simpler than your approach based on build scripts. It does not add the cc build dependency to the most popular build target. It allows to launch binaries compiled on a "modern" Linux hosts on older Linux distributions and vice versa.

[tamird]

It allows to launch binaries compiled on a "modern" Linux hosts on older Linux distributions and vice versa.

Remember that this code is used only on Android and MUSL. For Android, you always use the NDK, so this doesn't apply. In MUSL, support for getrandom was added 7 years ago. Who is building with a 7 year old MUSL?

The benefits of this approach are that it's making the resulting code faster (no indirect call), the implementation in linux_android_with_fallback.rs much simpler, and inlines the fallback code on systems where getrandom is statically not available.

I don't understand why the build dependency on cc is a meaningful downside.

[newpavlov]

You are forgetting about GNU targets which are much more popular than MUSL and Android targets combined. With the current approach I can compile my project on RHEL 7 and get a binary which properly uses the getrandom syscall on modern distributions, while with your approach my application will be locked to the file fallback. It also applies to the reverse scenario (there are some quirks around glibc symbol versioning, but it's not relevant to this crate), with your approach I would get a binary which links to the getrandom symbol and would fail with a linking error on an old distribution.

making the resulting code faster (no indirect call),

As I wrote here, this is a vanishingly small difference in our case. And if you care about it or if you find dlsym problematic for some reason, you can switch to the linux_getrandom opt-in backend.

I don't understand why the build dependency on cc is a meaningful downside.

getrandom is a pretty fundamental dependency and some users are quite sensitive to additional dependencies even if they are also "fundamental", e.g. see the recent debacle around use of zerocopy in rand and troubles we had with lock file bloat. cc also can be problematic because it requires installation of a C compiler which is not present by default on many distributions.

[tamird]

You are forgetting about GNU targets which are much more popular than MUSL and Android targets combined. With the current approach I can compile my project on RHEL 7 and get a binary which properly uses the getrandom syscall on modern distributions, while with your approach my application will be locked to the file fallback. It also applies to the reverse scenario (there are some quirks around glibc symbol versioning, but it's not relevant to this crate), with your approach I would get a binary which links to the getrandom symbol and would fail with a linking error on an old distribution.

Ah! I missed the fact that this implementation is used on GNU (on some architectures).

I don't understand why the build dependency on cc is a meaningful downside.

getrandom is a pretty fundamental dependency and many users are quite sensitive to additional dependencies even if they are also "fundamental", e.g. see the recent debacle around use of zerocopy in rand and troubles we had with lock file bloat. cc is also can be problematic because it requires installation of a C compiler which is not present by default on many distributions.

Indeed, this makes more sense when we talk about GNU targets (Android is always cross-compiled). I agree there's no way to support GNU targets without dlsym or weak linking, which isn't possible on nightly.