Cross compiling on Linux is untested, undocumented, and broken

[tiran]

BPO	45881
Nosy	@brettcannon, @tiran, @moreati
PRs	bpo-45881: Use CC from env first for cross building (GH-29752) #29752 [3.10] bpo-45881: Use CC from env first for cross building (GH-29752). (GH-29753) #29753 [3.9] bpo-45881: Use CC from env first for cross building (GH-29752) (GH-29754) #29754 bpo-45881: configure --with-freeze-module --with-build-python (GH-29835) #29835
Dependencies	bpo-45886: Fix Program/_freeze_module for cross compiling Python bpo-45909: sysconfig --generate-posix-vars creates wrong file when cross compiling

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-11-23.16:37:45.271>
labels = ['3.9', '3.10', '3.11', 'build', 'tests', 'docs']
title = 'Cross compiling on Linux is untested, undocumented, and broken'
updated_at = <Date 2021-11-29.16:23:38.440>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2021-11-29.16:23:38.440>
actor = 'christian.heimes'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Build', 'Documentation', 'Tests', 'Cross-Build']
creation = <Date 2021-11-23.16:37:45.271>
creator = 'christian.heimes'
dependencies = ['45886', '45909']
files = []
hgrepos = []
issue_num = 45881
keywords = ['patch']
message_count = 7.0
messages = ['406853', '406860', '406935', '406942', '407016', '407165', '407292']
nosy_count = 4.0
nosy_names = ['brett.cannon', 'christian.heimes', 'docs@python', 'Alex.Willmer']
pr_nums = ['29752', '29753', '29754', '29835']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue45881'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[tiran]

Cross compiling is an approach to compile a program for a different CPU architecture and platform, e.g. compile for an ARM64 (aarch64) or WASM on a x86_64 build system.

Python configure script, Makefile, and setup.py have multiple references to cross compiling. However I could not find any documentation in the devguide or Python docs how to cross compile. We also lack CI (buildbot) to test cross compiling. This lack of awareness and testing leads to breakage of the feature. For example the design of Programs/_freeze_module in main (3.11-dev) is incompatible with cross compiling.

I kinda got cross compiling working with 3.10, but only with some additional hacks and patches. I also ran into other problems like _PYTHON_HOST_PLATFORM env var is not automatically forwarded to setup.py. The helper functions add_multiarch_paths() and add_cross_compiling_paths() break builds for me, too. Cross compiling only works when the methods are commented out.