Add rpath to link command in Test::Alien by hakonhagland · Pull Request #415 · PerlAlien/Alien-Build

hakonhagland · 2024-07-15T20:10:50Z

I am trying to build a shared libgsl.so with Alien::GSL, see PerlAlien/Alien-GSL#17 for more information. When using an alienfile with:

 build [
[ './configure --prefix=%{.install.prefix} --with-pic --enable-shared --disable-static' ],
[ '%{make}' ],
[ '%{make} install' ],
];

Test::Alien::xs_ok() gives following error:

[...]
t/alien_gsl.t .. 2/? # Failed test 'xs'
# at t/alien_gsl.t line 11.
# XSLoader failed
# Can't load '/home/dockeruser/Alien-GSL-Shared/_alien/tmp/test-alien-0zwdsb/auto/Test/Alien/XS/Mod0AAAA/Mod0AAAA.so' for module Test::Alien::XS::Mod0AAAA: libgsl.so.28: cannot open shared object file: No such file or directory at /usr/lib/x86_64-linux-gnu/perl-base/DynaLoader.pm line 201.
# at /home/dockeruser/perl5/lib/perl5/Test/Alien.pm line 483.
# Compilation failed in require at /home/dockeruser/perl5/lib/perl5/Test/Alien.pm line 483.
# BEGIN failed--compilation aborted at /home/dockeruser/perl5/lib/perl5/Test/Alien.pm line 483.
t/alien_gsl.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/3 subtests [...]

I believe we need to pass the rpath to the linker command to fix this.

plicease · 2024-07-16T16:12:13Z

What you want for this is to use Alien::Role::Dino with your Alien, which knows what to do on the various different platforms.

hakonhagland · 2024-07-16T17:42:56Z

Thanks for the tip. I tested now Alien::Role::Dino here: hakonhagland/Math-GSL-Alien#1, but it still uses configure --disable-shared. Maybe I still need to use a custom build in the shared block in the alienfile, for example:

 build [
[ './configure --prefix=%{.install.prefix} --with-pic --enable-shared --disable-static' ],
[ '%{make}' ],
[ '%{make} install' ],
];

instead of using the plugin 'Build::Autoconf' ?

hakonhagland · 2024-07-16T18:16:13Z

Maybe I still need to use a custom build in the shared block in the alienfile

Yes that seems to work:)

@plicease If you have time, could you review the module Alien::GSL::Shared, currently at GitHub only: https://github.com/hakonhagland/Alien-GSL-Shared/tree/main

Can this module be published to CPAN as it is?

hakonhagland · 2024-07-17T14:33:42Z

Maybe I still need to use a custom build in the shared block in the alienfile
... Yes that seems to work:)

It works on Linux and macOS, but on Windows I think we still need something similar to plugin 'Build::Autoconf' which uses Alien::Build::Plugin::Build::Autoconf and does many things behind the scenes to make it work on Windows, right?

hakonhagland · 2024-07-17T18:56:50Z

on Windows I think we still need something similar to plugin 'Build::Autoconf'

If I combine plugin 'Build::Autoconf' and a custom build rule it still does not work on Windows. Here is the alienfile:

use alienfile;
plugin 'PkgConfig' => (pkg_name => 'gsl');
share {
start_url 'https://ftp.gnu.org/gnu/gsl';
plugin Download => (
filter => qr/^gsl-([\d\.]+)\.tar\.gz$/,
version => qr/^gsl-([\d\.]+)\.tar\.gz$/,
prefer => 1,
);
plugin Extract => 'tar.gz';
plugin 'Build::Autoconf';
build [
[ './configure --prefix=%{.install.prefix} --with-pic --enable-shared --disable-static' ],
[ '%{make}' ],
[ '%{make} install' ],
];
plugin 'Gather::Dino';
};

When using this alienfile on windows, it will still try to run ./configure ... (which does not work) instead of using (the correct) MSYS sh.exe to run configure like this: sh configure .... Any idea why this line

Alien-Build/lib/Alien/Build/Plugin/Build/Autoconf.pm

Line 181 in 96cdc77

$configure = _win ? 'sh ./configure' : './configure';

is not executed?

hakonhagland · 2024-07-17T20:55:28Z

it will still try to run ./configure ... (which does not work)

I think we need to use %{configure} instead of ./configure, so the following seems to work:

 build [
[ '%{configure} --prefix=%{.install.prefix} --with-pic --enable-shared --disable-static' ],
[ '%{make}' ],
[ '%{make} install' ],
];

hakonhagland · 2024-07-17T21:22:06Z

Now "configure" and "make" succeeds on Windows, but "make install" fails:

Alien::Build::CommandSequence> + make install
make[1]: Entering directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8'
Making install in gsl
make[2]: Entering directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/gsl'
make[3]: Entering directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/gsl'
make[3]: Nothing to be done for `install-exec-am'.
make[3]: Nothing to be done for `install-data-am'.
make[3]: Leaving directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/gsl'
make[2]: Leaving directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/gsl'
Making install in utils
make[2]: Entering directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/utils'
make[3]: Entering directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/utils'
make[3]: Nothing to be done for `install-exec-am'.
make[3]: Nothing to be done for `install-data-am'.
make[3]: Leaving directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/utils'
make[2]: Leaving directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/utils'
Making install in sys
make[2]: Entering directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/sys'
make[3]: Entering directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/sys'
make[3]: Nothing to be done for `install-exec-am'.
/usr/bin/mkdir -p 'C:/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/destdir_Ryu0C:/STRAWB~1/perl/site/lib/auto/share/dist/Math-GSL-Alien/include/gsl'
/usr/bin/mkdir: cannot create directory `C:/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/destdir_Ryu0C:': No such file or directory
make[3]: *** [install-pkgincludeHEADERS] Error 1
make[3]: Leaving directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/sys'
make[2]: *** [install-am] Error 2
make[2]: Leaving directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8/sys'
make[1]: *** [install-recursive] Error 1
make[1]: Leaving directory `/c/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/build_k177/gsl-2.8'
external command failed at C:/Strawberry/perl/vendor/lib/Alien/Build/CommandSequence.pm line 73.
gmake: *** [makefile:1033: _alien/mm/build] Error 2

Notice the command:

 /usr/bin/mkdir -p 'C:/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/destdir_Ryu0C:/STRAWB~1/perl/site/lib/auto/share/dist/Math-GSL-Alien/include/gsl'

First, "/usr/bin/mkdir" does not look correct on Windows, and second, the path seems to be made up of two separate paths starting with "C:/"

hakonhagland · 2024-07-17T22:22:10Z

but "make install" fails:

We need to remove --prefix when using %{configure} or else it will appear two times on the command line. So the following seems to work:

 build [
[ '%{configure} --enable-shared --disable-static' ],
[ '%{make}' ],
[ '%{make} install' ],
];

hakonhagland · 2024-07-18T08:30:47Z

It is working, but in some cases I think the path names becomes too long since "make install" seems to use the absolute path when concatenating two paths instead of using one absolute and one relative path. For example:

make[3]: Nothing to be done for `install-exec-am'.
/usr/bin/mkdir -p 'C:/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/destdir_BgN5/C/STRAWB~1/perl/site/lib/auto/share/dist/Math-GSL-Alien/include/gsl'

Shouldn't this be rather:

mkdir -p 'C:/Users/hakon/perl/cpan-modules/Math-GSL-Alien-1.03/_alien/destdir_BgN5/include/gsl

?

shawnlaffan · 2024-07-18T08:48:39Z

It worked without issue when I tried a few hours ago. I'm running it again to be sure as I hand-edited the alienfile from the github repo.

This is using Strawberry Perl portable, 5.38.2.2, and env var ALIEN_INSTALL_TYPE=share

hakonhagland · 2024-07-18T08:59:18Z

seems to use the absolute path when concatenating two paths

It seems to be the correct behavior to concatenate absolute path? I tested on linux now, and it uses absolute paths there too. However, the path length can the become a problem on windows, right? Isn't there a limit on path length like 260 characters?

hakonhagland · 2024-07-18T08:59:57Z

It worked without issue when I tried a few hours ago

@shawnlaffan Thanks for testing :)

hakonhagland · 2024-07-18T09:06:51Z

When testing from a GitHub action, I get a path the is 264 characters and it fails. See:

https://github.com/hakonhagland/perl-math-gsl/actions/runs/9987653501/job/27602620644#step:7:13310

The error is:

/usr/bin/install: writing `C:/HOSTED~1/windows/STRAWB~1/532~1.1/x64/data/.cpanm/work/1721289541.5656/Math-GSL-Alien-1.05/_alien/destdir_bYvy/C/hostedtoolcache/windows/strawberry-perl/5.32.1/x64/perl/site/lib/auto/share/dist/Math-GSL-Alien/include/gsl/gsl_permute_matrix_complex_long_double.h': Invalid request code

shawnlaffan · 2024-07-18T09:26:06Z

You might be hitting clashes with GH Actions putting the Git for Windows MSYS bin dir in the path.

I've had to work around these in the past using an around hook, e.g.:
https://github.com/shawnlaffan/perl-alien-gdal/blob/c458e49b768bd694e23fdd2fe1322063b4619e06/alienfile#L303
https://github.com/shawnlaffan/perl-alien-gdal/blob/c458e49b768bd694e23fdd2fe1322063b4619e06/alienfile#L594-L630

shawnlaffan · 2024-07-18T10:07:41Z

It worked without issue when I tried a few hours ago
@shawnlaffan Thanks for testing :)

Confirming it builds and passes tests on my local system.

hakonhagland · 2024-07-18T17:14:28Z

Confirming it is a problem with the MSYS install.exe program's handling of long path names. On my machine with Windows 11, from PowerShell: Long paths are enabled:

> reg query HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem /v LongPathsEnabled
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
LongPathsEnabled REG_DWORD 0x1

However, running a test perl script that executes install.exe with varying path lengths fails when the path lengths approaches 260 characters.

hakonhagland · 2024-07-18T17:58:21Z

it is a problem with the MSYS install.exe

I have also installed MSYS2 in C:\msys2 and C:\msys2\usr\bin\install.exe does not have any problems with long path names.. So maybe we should use this install.exe instead?

shawnlaffan · 2024-07-18T21:43:18Z

I suspect path lengths are a symptom and not the root cause. The path you posted has the first half using windows form and the second using MSYS form. This is why I suspect the MSYS (or some other) shell is being passed the args.

Currently it refers to C:/HOSTED~1/windows/... to which C/hostedtoolcache/windows/... is concatenated. These are both the same location and should not be concatenated.

The install path should read as

C:/hostedtoolcache/windows/strawberry-perl/5.32.1/x64/perl/site/lib/auto/share/dist/Math-GSL-Alien/include/gsl/

hakonhagland · 2024-07-21T06:04:31Z

The path you posted has the first half using windows form and the second using MSYS form

@shawnlaffan Right, but I tested now on Linux and the same problem exists there, even with the original Alien::GSL module (not Math::GSL::Alien as we are discussing in this thread). For example (referring to this step) I get output like this on linux:

 /usr/bin/install -c -m 644 gsl_eigen.h '/home/dockeruser/Alien-GSL-1.07/_alien/destdir__DHH/home/dockeruser/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/auto/share/dist/Alien-GSL/include/gsl'

Notice the two parts: /home/dockeruser/Alien-GSL-1.07/_alien/destdir__DHH and /home/dockeruser/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/auto/share/dist/Alien-GSL/include/gsl

shawnlaffan · 2024-07-21T08:06:54Z

I'm seeing the same path doubling on my machine when using WSL but the tests and installation work without issue.

There is this log entry, though:

Alien::Build::Plugin::Core::Gather> mirror /home/user/.cpanm/work/1721548601.912/Math-GSL-Alien-1.05/_alien/destdir_EbYM/home/user/perl5/perlbrew/perls/perl-5.36.0/lib/site_perl/5.36.0/x86_64-linux/auto/share/dist/Math-GSL-Alien => /home/shawn/.cpanm/work/1721548601.912/Math-GSL-Alien-1.05/blib/lib/auto/share/dist/Math-GSL-Alien

shawnlaffan · 2024-07-21T08:12:02Z

My untested guess is that something odd is happening in this sub and the paths are not being combined, or the paths passed to this sub are not as expected:

Alien-Build/lib/Alien/Build/Util.pm

Lines 120 to 125 in 96cdc77

sub_destdir_prefix
 { 
my($destdir, $prefix) = @_; 
$prefix =~ s{^/?([a-z]):}{$1}iif $^O eq'MSWin32'; 
 path($destdir)->child($prefix)->stringify; 
 } 

.

Add rpath to link command
bb69e8c

hakonhagland mentioned this pull request Jul 21, 2024
Upgrade to MSYS2? PerlAlien/Alien-MSYS#25
Open

Uh oh!

Conversation

hakonhagland commented Jul 15, 2024

Uh oh!

plicease commented Jul 16, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hakonhagland commented Jul 16, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hakonhagland commented Jul 16, 2024

Uh oh!

hakonhagland commented Jul 17, 2024

Uh oh!

hakonhagland commented Jul 17, 2024

Uh oh!

hakonhagland commented Jul 17, 2024

Uh oh!

hakonhagland commented Jul 17, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hakonhagland commented Jul 17, 2024

Uh oh!

hakonhagland commented Jul 18, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shawnlaffan commented Jul 18, 2024

Uh oh!

hakonhagland commented Jul 18, 2024

Uh oh!

hakonhagland commented Jul 18, 2024

Uh oh!

hakonhagland commented Jul 18, 2024

Uh oh!

shawnlaffan commented Jul 18, 2024

Uh oh!

shawnlaffan commented Jul 18, 2024

Uh oh!

hakonhagland commented Jul 18, 2024

Uh oh!

hakonhagland commented Jul 18, 2024

Uh oh!

shawnlaffan commented Jul 18, 2024

Uh oh!

hakonhagland commented Jul 21, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shawnlaffan commented Jul 21, 2024

Uh oh!

shawnlaffan commented Jul 21, 2024

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

plicease commented Jul 16, 2024•
edited
Loading

hakonhagland commented Jul 16, 2024•
edited
Loading

hakonhagland commented Jul 17, 2024•
edited
Loading

hakonhagland commented Jul 18, 2024•
edited
Loading

hakonhagland commented Jul 21, 2024•
edited
Loading