Integrating with LuaRocks

[tilkinsc]

This issue serves for discussion on integrating LuaRocks with Lua.NET.

[tilkinsc]

I use the standard headers provided by lua and luajit. So this makes it compatible with whatever luarocks is doing to build and link. I am following this guide: https://github.com/luarocks/luarocks/wiki/Installation-instructions-for-Unix Lua doesn't build correctly using their script because the lua tar.gz moved.

LuaRocks requires an interpreter to run for tasks such as downloading rocks. Download and install lua. I am using a custom directory here because I dont like installing non-aur programs into my system.

mkdir lua-install
curl https://lua.org/ftp/lua-5.4.6.tar.gz > lua-5.4.6.tar.gz
tar xzf lua-5.4.6.tar.gz
pushd lua-5.4.6
make -j24
make INSTALL_TOP="../lua-install" install
popd

Next up is configuring and installing LuaRocks. I am using hard copied paths below that will only work for me. However, beyond the home /home/mkp/src/Lua.NET prefix, you want to target the relevant directories. This configures it so that it will use the lua interpreter installed above and keep the rocks tree separate. This will be handy for giving to liblua when requiring.

git clone https://github.com/luarocks/luarocks
mkdir luarocks_install
pushd luarocks
./configure \
 --with-lua-include=/home/mkp/src/Lua.NET/lua-install/include/ \
 --prefix=/home/mkp/src/Lua.NET/luarocks_install \
 --lua-version=5.4 \
 --with-lua-bin=/home/mkp/src/Lua.NET/lua-install/bin \
 --rocks-tree=/home/mkp/src/Lua.NET/luarocks_install/rocks
popd

Question now is how to hook everything up with this sane install. require() will search various directories. But first we need a package installed.

[mkp@arch luarocks_install]$ bin/luarocks install luasocket
Installing https://luarocks.org/luasocket-3.1.0-1.src.rock

luasocket 3.1.0-1 depends on lua >= 5.1 (5.4-1 provided by VM: success)
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/mime.c -o src/mime.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/compat.c -o src/compat.o -DLUASOCKET_DEBUG
gcc -shared -o /tmp/luarocks_build-LuaSocket-3.1.0-1-9519233/mime/core.so src/mime.o src/compat.o
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/luasocket.c -o src/luasocket.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/timeout.c -o src/timeout.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/buffer.c -o src/buffer.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/io.c -o src/io.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/auxiliar.c -o src/auxiliar.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/options.c -o src/options.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/inet.c -o src/inet.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/except.c -o src/except.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/select.c -o src/select.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/tcp.c -o src/tcp.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/udp.c -o src/udp.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/compat.c -o src/compat.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/usocket.c -o src/usocket.o -DLUASOCKET_DEBUG
gcc -shared -o /tmp/luarocks_build-LuaSocket-3.1.0-1-9519233/socket/core.so src/luasocket.o src/timeout.o src/buffer.o src/io.o src/auxiliar.o src/options.o src/inet.o src/except.o src/select.o src/tcp.o src/udp.o src/compat.o src/usocket.o
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/buffer.c -o src/buffer.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/compat.c -o src/compat.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/auxiliar.c -o src/auxiliar.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/options.c -o src/options.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/timeout.c -o src/timeout.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/io.c -o src/io.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/usocket.c -o src/usocket.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/unix.c -o src/unix.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/unixdgram.c -o src/unixdgram.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/unixstream.c -o src/unixstream.o -DLUASOCKET_DEBUG
gcc -shared -o /tmp/luarocks_build-LuaSocket-3.1.0-1-9519233/socket/unix.so src/buffer.o src/compat.o src/auxiliar.o src/options.o src/timeout.o src/io.o src/usocket.o src/unix.o src/unixdgram.o src/unixstream.o
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/buffer.c -o src/buffer.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/compat.c -o src/compat.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/auxiliar.c -o src/auxiliar.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/options.c -o src/options.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/timeout.c -o src/timeout.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/io.c -o src/io.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/usocket.c -o src/usocket.o -DLUASOCKET_DEBUG
gcc -O2 -fPIC -I/home/mkp/src/Lua.NET/lua-install/include -c src/serial.c -o src/serial.o -DLUASOCKET_DEBUG
gcc -shared -o /tmp/luarocks_build-LuaSocket-3.1.0-1-9519233/socket/serial.so src/buffer.o src/compat.o src/auxiliar.o src/options.o src/timeout.o src/io.o src/usocket.o src/serial.o
No existing manifest. Attempting to rebuild...
luasocket 3.1.0-1 is now installed in /home/mkp/src/Lua.NET/luarocks_install/rocks (license: MIT)

LuaRocks builds shared libraries and the entry point is typically core.{so,dll} However, we do know that it lobbed everything in luarocks_install/rocks.

[mkp@arch luarocks_install]$ tree rocks
rocks
├── lib
│ ├── lua
│ │ └── 5.4
│ │ ├── mime
│ │ │ └── core.so
│ │ └── socket
│ │ ├── core.so
│ │ ├── serial.so
│ │ └── unix.so
│ └── luarocks
│ └── rocks-5.4
│ ├── luasocket
│ │ └── 3.1.0-1
│ │ ├── docs
│ │ │ ├── dns.html
│ │ │ ├── ...
│ │ ├── etc
│ │ │ ├── b64.lua
│ │ │ ├── check-links.lua
│ │ │ ├── check-memory.lua
│ │ │ ├── cookie.lua
│ │ │ ├── dict.lua
│ │ │ ├── dispatch.lua
│ │ │ ├── eol.lua
│ │ │ ├── forward.lua
│ │ │ ├── get.lua
│ │ │ ├── links
│ │ │ ├── lp.lua
│ │ │ ├── qp.lua
│ │ │ ├── README
│ │ │ └── tftp.lua
│ │ ├── luasocket-3.1.0-1.rockspec
│ │ ├── rock_manifest
│ │ ├── samples
│ │ │ ├── cddb.lua
│ │ │ ├── daytimeclnt.lua
│ │ │ ├── echoclnt.lua
│ │ │ ├── echosrvr.lua
│ │ │ ├── listener.lua
│ │ │ ├── lpr.lua
│ │ │ ├── mclisten.lua
│ │ │ ├── mcsend.lua
│ │ │ ├── README
│ │ │ ├── talker.lua
│ │ │ └── tinyirc.lua
│ │ └── test
│ │ ├── auth
│ │ │ └── index.html
│ │ ├── cgi
│ │ │ ├── cat
│ │ │ ├── cat-index-html
│ │ │ ├── env
│ │ │ ├── query-string
│ │ │ ├── redirect-loop
│ │ │ └── request-uri
│ │ ├── dicttest.lua
│ │ ├── excepttest.lua
│ │ ├── find-connect-limit
│ │ ├── ftptest.lua
│ │ ├── hello.lua
│ │ ├── httptest.lua
│ │ ├── index.html
│ │ ├── ltn12test.lua
│ │ ├── luasocket.png
│ │ ├── mimetest.lua
│ │ ├── README
│ │ ├── smtptest.lua
│ │ ├── stufftest.lua
│ │ ├── tcp-getoptions
│ │ ├── test_bind.lua
│ │ ├── testclnt.lua
│ │ ├── test_getaddrinfo.lua
│ │ ├── testmesg.lua
│ │ ├── test_socket_error.lua
│ │ ├── testsrvr.lua
│ │ ├── testsupport.lua
│ │ ├── tftptest.lua
│ │ ├── udpconnectclnt.lua
│ │ ├── udpconnectsrvr.lua
│ │ ├── udp-zero-length-send
│ │ ├── udp-zero-length-send-recv
│ │ ├── unixdgramclnt.lua
│ │ ├── unixdgramsrvr.lua
│ │ ├── unixstreamclnt.lua
│ │ ├── unixstreamsrvr.lua
│ │ ├── upload.html
│ │ ├── urltest.lua
│ │ ├── utestclnt.lua
│ │ └── utestsrvr.lua
│ └── manifest
└── share
 └── lua
 └── 5.4
 ├── ltn12.lua
 ├── mime.lua
 ├── socket
 │ ├── ftp.lua
 │ ├── headers.lua
 │ ├── http.lua
 │ ├── smtp.lua
 │ ├── tp.lua
 │ └── url.lua
 └── socket.lua

Looking at the cookie.lua example it appears that libraries are required from the rocks/lib/lua/5.4 rocks/share/lua/5.4 directory.

local socket = require"socket" -- rocks/lib/lua/5.4/socket/core.so
local http = require"socket.http" -- share/lua/5.4/socket/http.lua
local url = require"socket.url" -- share/lua/5.4/socket/url.lua

This means that as long as we 1. target the appropriate lua version (luarocks supports multi-version), 2. link the appropriate lua dll to the resulting binaries (enabled through config), 3. provide lua the rocks install paths as mentioned in the above paragraph, we should be able to integrate luarocks seemlessly.

To resolve 1 and 3, this will need to be handled in-code. This is the print-out of the package in our _G when we luaL_openlibs(L);

[mkp@arch Lua.NET]$ lua-install/bin/lua
Lua 5.4.6 Copyright (C) 1994-2023 Lua.org, PUC-Rio
> for i, v in next, package do print(i, v) end
cpath /usr/local/lib/lua/5.4/?.so;/usr/local/lib/lua/5.4/loadall.so;./?.so
searchpath function: 0x6031b0adaf30
path /usr/local/share/lua/5.4/?.lua;/usr/local/share/lua/5.4/?/init.lua;/usr/local/lib/lua/5.4/?.lua;/usr/local/lib/lua/5.4/?/init.lua;./?.lua;./?/init.lua
searchers table: 0x6031b23118a0
loaded table: 0x6031b2310a60
config /
;
?
!
-

preload table: 0x6031b2311be0
loadlib function: 0x6031b0adb160

We are using a non-standard /usr/local/share/lua/5.4 directory, so for path you will need to concat this string with the share directory something like /absolute/path/to/rocks/share/lua/5.4/?.lua;. For cpath, you will need to do something like /absolute/path/to/rocks/lib/lua/5.4/socket/?.so; Im not sure how package.searchers work, but that might be another entrypoint to doing this.

using LuaNET.Lua54;
using static LuaNET.Lua54.Lua;

lua_State L = luaL_newstate();
if (L == 0)
{
	Console.WriteLine("Unable to create context!");
}
luaL_openlibs(L);
// TODO: concat package.cpath package.path with above information
lua_pushcfunction(L, lfunc);
Console.WriteLine(lua_pcall(L, 0, 0, 0));
lua_close(L);

To resolve 2, we can use

link_lua_explicitly (boolean) - Link the Lua library to the built modules when using the builtin mode (this is set to true for Cygwin).

This resolves the symbols for us, as the rocks built for us are not linked to anything, theyre just configured with symbols they could import via the headers we suppied with the lua reply we built above. However, it wont be linked to Lua.NET's shared libraries. This can also be configured through the config. Somehow. I would recommend you do so, so that you dont have to lug around yet another shared library in addition to the ones I provide with Lua.NET automatically.

[tilkinsc]

A REPL environment is NOT needed for LuaRocks. Just something able to execute a script with param args. So some internal solution could be resolved for dependency management as well.