Python build should check CPATH, C_INCLUDE_PATH for module dependencies

[JanKanis]

BPO	21571
Nosy	@ronaldoussoren, @ned-deily

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-05-24.21:29:37.914>
labels = ['type-feature', 'build']
title = 'Python build should check CPATH, C_INCLUDE_PATH for module dependencies'
updated_at = <Date 2015-04-03.18:23:18.970>
user = 'https://bugs.python.org/JanKanis'

bugs.python.org fields:

activity = <Date 2015-04-03.18:23:18.970>
actor = 'yesimon'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2014-05-24.21:29:37.914>
creator = 'JanKanis'
dependencies = []
files = []
hgrepos = []
issue_num = 21571
keywords = []
message_count = 4.0
messages = ['219056', '219071', '219384', '222163']
nosy_count = 4.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'JanKanis', 'yesimon']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue21571'
versions = ['Python 3.5']

[JanKanis]

When building, pythons setup.py tries to find external sources for optional modules such as ssl, sqlite, etc. For that it searches the CFLAGS environment variable for -I options. C compilers such as gcc and clang also interpret CPATH and C_INCLUDE_PATH as extra search paths for headers/sources. Setup.py should do the same and also look in paths specified in those environment variables.

[ned-deily]

setup.py already does that but only for builds on OS X (darwin); see http://hg.python.org/cpython/file/default/setup.py#l539. The comments there suggest to me that the code was made conditional to OS X to avoid introducing regressions in builds on other platforms. The restriction could be removed but probably only for a feature release (e.g. 3.5). Another approach that has been suggested is to add either configure options (like in bpo-21541) or environment variables (bpo-5575). In theory, the best approach might be to use the customization features provided by Modules/Setup*; unfortunately, at the moment there are some major problems with that in current Python 3.4 releases (see bpo-17095 for a proposed patch for 3.4).

[ronaldoussoren]

I'm pretty sure that I wrote the code Ned refers to, and that's indeed only targeting darwin to avoid breaking other platforms. That code could easily be made actively globally though, the only reason I didn't do so at the time is we were still getting used to the fact that MacOS used the same build infrastructure as other major platforms and hence were overly cautious.

Note that this doesn't do the same thing as Jan requests: GCC1 and clang2 can add additional directories to their header file search path using environment variables.

For Python's setup.py file the following are important:

• C_INCLUDE_PATH: a list of directories that are handled as if they are present at the end of the list of -isystem options (that is, system include files at a lower priority that the user provided ones)

• CPATH: simular, but for the '-I' option

• LIBRARY_PATH: similar, but for the '-L' option

All of them have a syntax similar to $PATH.

A patch to add support for these variables (and enables the handling of -I and -L for other platforms than darwin) should be easy enough, but I agree with Ned that this would be a new feature because it could break existing build systems (that is, building Python with this patch could result in a different build than without the patch due to the build picking up more or different external libraries).

Jan: are you willing to write such a patch? And if so, are you willing to sign a contributor agreement?

[JanKanis]

I can write a patch. I haven't signed a contributor agreement but I have no problem doing so. I am not sure when I will have time to write a patch though, so it could take some time.

[serhiy-storchaka]

setup.py no longer used to build CPython since 3.12 (see #94474).