bpo-40280: Add wasm cross build targets (GH-29771)

[tiran]

CPython uses the same target triplet convention as Rust:

$ rustc --print target-list | grep wasm
wasm32-unknown-emscripten
wasm32-unknown-unknown
wasm32-wasi
wasm64-unknown-unknown

Signed-off-by: Christian Heimes christian@python.org

https://bugs.python.org/issue40280

[emmatyping]

FWIW, this is enough to allow me to compile CPython with emscripten with the following settings:

# assuming emsdk, llvm, etc is installed...
READELF=llvm-readelf-12 emconfigure ./configure ac_cv_func_memfd_create=no ac_cv_func_eventfd=no ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no cross_compiling=yes --with-pydebug --with-suffix=.wasm --with-ensurepip=no --disable-ipv6 --host=wasm32-unknown-emscripten --build=x86_64-unknown-linux-gnu
emmake make FREEZE_MODULE=../cpython-build/Programs/_freeze_module

However it seems the built python.wasm won't run with any of the webassembly runtimes I tried, I think the finall emcc call would need to be tweaked...

[tiran]

I'm facing the same issue, e.g. wasmtime complaints about missing clock_gettime.

I fixed the sigset_t issue in GH-29770 and the wrong CC for cross compilation in GH-29752. Pyodide and similar projects have more downstream patches.

Did you know that you can put all custom autoconf overrides into a config.site file? I'm using this config.site file

# config.site-wasm
ac_cv_buggy_getaddrinfo=no
ac_cv_file__dev_ptmx=yes
ac_cv_file__dev_ptc=no
# undefined symbols in wasm:
ac_cv_func_posix_spawn=no
ac_cv_func_posix_spawnp=no
ac_cv_func_eventfd=no

to run CONFIG_SITE=config.site-wasm emconfigure ./configure ....

[brettcannon]

I don't know if the current solution will work with wasm32-wasi. We should also make sure to support wasm32-unknown-wasi.

[bedevere-bot]

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

[brettcannon]

Do we want to record all the incantations we are trying to make builds work somewhere? E.g. https://bugs.python.org/issue40280 or somewhere else like a GitHub org/repo (I'm happy to create a cpython-wasm repo and give you both write access to write down what does and doesn't work)?

@ethanhs while I think--with-suffix .wasm should trigger STANDALONE_WASM=1, I'm not sure, so you may need to set that environment variable to force emscripten to do a WASI build over an emscripten/browser/JS build.

[tiran]

configure runs the host and build strings through config.sub, which converts any short forms into long canonical form:

$ emconfigure ../../configure -C --host=wasm32-wasi --build=x86_64-unknown-linux-gnu
configure: ../../configure -C --host=wasm32-wasi --build=x86_64-unknown-linux-gnu
configure: loading site script /usr/local/etc/config.site
configure: loading cache config.cache
checking for git... found
checking build system type... x86_64-unknown-linux-gnu
checking host system type... wasm32-unknown-wasi

--host=wasm32-wasi -> wasm32-unknown-wasi

For more information see:

• https://wiki.osdev.org/Target_Triplet
• https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Specifying-Target-Triplets.html

[emmatyping]

Do we want to record all the incantations we are trying to make builds work somewhere? E.g. https://bugs.python.org/issue40280 or somewhere else like a GitHub org/repo (I'm happy to create a cpython-wasm repo and give you both write access to write down what does and doesn't work)?

This seems like a great idea! Plus there will definitely need to be more patches for WASI, and probably Emscripten as well, so we can coordinate there if needed.

while I think --with-suffix .wasm should trigger STANDALONE_WASM=1, I'm not sure, so you may need to set that environment variable to force emscripten to do a WASI build over an emscripten/browser/JS build.

Certainly could! I was mostly seeing if I could get things to build at all before I went to bed. I'll probably play with things more tomorrow (and hopefully have something to show for it).

[tiran]

+1 for Brett's proposal.

Progress! I'm now getting a different error from wasmtime. My emscripten version may not support pthread.

# wasmtime ./python.wasm
Error: failed to run main module `./python.wasm`
Caused by:
0: failed to instantiate "./python.wasm"
1: unknown import: `env::pthread_create` has not been defined

[emmatyping]

Progress! I'm now getting a different error from wasmtime. My emscripten version may not support pthread.

Great! Unfortunately I don't think wasmtime supports pthreads. The only WASM runtime that I could find that did was https://github.com/bytecodealliance/wasm-micro-runtime

Wasmer seems to stub pthread out https://github.com/wasmerio/wasmer/blob/master/lib/emscripten/src/pthread.rs, but maybe it just passes those through?