add installation support for pkg-config dependency detection by eli-schwartz · Pull Request #4077 · pybind/pybind11

eli-schwartz · 2022-07-17T16:00:28Z

pkg-config is a build-system agnostic alternative to pybind11Config.cmake that can be used from build systems other than cmake.

Fixes#230

Suggested changelog entry:

* Include a pkg-config file when installing pybind11, such as in the Python package.

eli-schwartz · 2022-07-17T16:02:57Z

- endforeach()
- set(${joined_path} "${temp_path}" PARENT_SCOPE)
+ set(temp_path "${first_path_segment}")
+ foreach(current_segment IN LISTS ARGN)


Is there some way to tell pre-commit.ci that this file belongs to another repo and shouldn't be autoformatted as that will result in a confusing file delta between here and the upstream version? 👀

Yes, you can add an exclude key regex to the pre-commit-config.yaml for the relevant hook.

Filename seems to have a dot in front of it...
This file is very big and I have basically zero knowledge of how it works other than "frustrating thing that messes up my commit messages", can I have a small hint at what I'm looking at, here? :)

https://github.com/pyodide/pyodide/blob/be1b3c2a6f3ae869300b5630f523dbfb15e3d2e3/.pre-commit-config.yaml#L1
is an example.

Also could be worth mentioning this file can be removed once we depend on CMake 3.20+.

Ah, thanks. Global exclude sounds helpful.

I added comments in CMakeLists.txt, marked with # TODO: for easy searchability.

henryiii · 2022-07-19T08:24:02Z

I'm traveling & at conferences, I'd like to take a look at the generated file. I might want to tweak the CMake code just a bit too. I'll have a bit more time starting Wednesday, so ping me ~Thursday if I don't get back to this by then. Thanks!

eli-schwartz · 2022-07-25T11:56:06Z

Ping. :)

I'd like to take a look at the generated file.

BTW, here is the generated code for the regular wheel:

$ bsdtar -xOf dist/pybind11-2.11.0.dev1-py3-none-any.whl pybind11/share/pkgconfig/pybind11.pc
prefix=${pcfiledir}/../../
includedir=${prefix}/include
Name: pybind11
Description: Seamless operability between C++11 and Python
Version: 2.11.0
Cflags: -I${includedir}

It's the same content for the pybind11_global wheel except, of course, the location.
File lists, more or less:

pybind11:

pybind11/share/cmake/pybind11/FindPythonLibsNew.cmake
pybind11/share/cmake/pybind11/pybind11Common.cmake
pybind11/share/cmake/pybind11/pybind11Config.cmake
pybind11/share/cmake/pybind11/pybind11ConfigVersion.cmake
pybind11/share/cmake/pybind11/pybind11NewTools.cmake
pybind11/share/cmake/pybind11/pybind11Targets.cmake
pybind11/share/cmake/pybind11/pybind11Tools.cmake
pybind11/share/pkgconfig/pybind11.pc

pybind11_global

pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/FindPythonLibsNew.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11Common.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11Config.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11ConfigVersion.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11NewTools.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11Targets.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11Tools.cmake
pybind11_global-2.11.0.dev1.data/data/share/pkgconfig/pybind11.pc

eli-schwartz · 2022-07-29T17:14:34Z

@henryiii ping

henryiii · 2022-07-29T18:39:23Z

Thanks! I'm back in Princeton now after nearly a month of travel.

Is there any reason it's joining the path rather than hardcoding the separator, as in ${pcfiledir}/../../? The generated file gets including in the wheel, and must be usable from all OS's (and ideally creatable on all OSs identically), so ${pcfiledir}\..\..\ would be invalid.

eli-schwartz · 2022-07-29T21:46:29Z

The JoinPaths function uses forward slashes anyway, exactly for portability reasons. And the paths need to be joined because cmake's includedir may or may not be an absolute path. (It will be a relative path when invoking cmake from setup.py, though.)

eli-schwartz · 2022-08-01T17:31:28Z

@henryiii the test that you added is failing due to platform differences between \r\n and \n. Not sure what to do about that, I do notice that the pybind11Config.cmake check only tries to assert that a set(...) exists in the file, it's very newline style agnostic.

Is this another of those "setuptools re-encodes all the newlines" issues? :(

eli-schwartz · 2022-08-05T18:25:42Z

By the way, merging this PR would help Void Linux successfully cross-compile SciPy. See scipy/scipy#16783

henryiii · 2022-08-05T18:55:04Z

Sorry, in the third of four conferences in a row. And preparation for CPython 3.10rc1 is taking free time too. This just needs to address the failing test. Not too worried about the line endings, happy to normalize them in the test. I hopefully can do it by Monday, ping me if I don’t! (Or you can do it if you want)

eli-schwartz · 2022-08-05T23:10:33Z

No problem, take your time. I won't have time before then either, anyway. :)

pkg-config is a buildsystem-agnostic alternative to `pybind11Config.cmake` that can be used from build systems other than cmake. Fixespybind#230

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

rwgk · 2022-08-08T21:22:17Z

This is totally over my head but I looked out of curiosity. Is this impression correct?

pkg-config is build-system agnostic, but the implementation of the pybind11 pkg-config support relies on cmake?

If that's true, it might be nice to add that as a note to the PR description.

Initially the phrase "alternative to pybind11Config.cmake" in the PR description led me to believe the pkg-config support will not involve cmake at all.

eli-schwartz · 2022-08-08T21:35:15Z

CMake is the build system that pybind11 uses, so CMake is used to generate the pkg-config file. Once generated (for example by producing a python wheel, or by running cmake && make install in a Linux distro) the resultant installable files are agnostic.

For the purposes of e.g. SciPy (and more generally meson) we don't involve CMake.

rwgk · 2022-08-08T22:17:36Z

Got it, thanks for the explanation!

eli-schwartz · 2022-08-10T03:00:21Z

@henryiii thanks for figuring out the CI. Really happy to see this landing. :)

eli-schwartz requested a review from henryiii as a code owner July 17, 2022 16:00

eli-schwartz commented Jul 17, 2022
View reviewed changes

eli-schwartz force-pushed the pkgconfig branch from 5413616 to d02bc0cCompare July 17, 2022 18:14

eli-schwartz mentioned this pull request Aug 5, 2022
BUG: meson fails to properly detect numpy, pybind11 and (sort of) pythran when cross compiling scipy/scipy#16783
Closed

henryiii mentioned this pull request Aug 8, 2022
[PLAN] Tracking issue for 2.10.1 #4125
Closed

eli-schwartzand others added 2 commits August 8, 2022 15:40

add installation support for pkg-config dependency detection
c933d9b
pkg-config is a buildsystem-agnostic alternative to `pybind11Config.cmake` that can be used from build systems other than cmake. Fixespybind#230

tests: add test for pkg config
fc48124
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

henryiii force-pushed the pkgconfig branch from b6a9357 to fc48124Compare August 8, 2022 19:58

henryiii approved these changes Aug 8, 2022
View reviewed changes

henryiii merged commit 5bdd3d5 into pybind:masterAug 9, 2022

github-actionsBot added the needs changelog Possibly needs a changelog entry label Aug 9, 2022

eli-schwartz deleted the pkgconfig branch August 10, 2022 02:59

henryiii removed the needs changelog Possibly needs a changelog entry label Oct 20, 2022

henryiii mentioned this pull request Oct 21, 2022
Scratch PR for testing pybind11 v2.10.0 with Python 3.11rc2 #4271
Closed

rwgk mentioned this pull request Feb 10, 2023
FWD pybind11 google/pybind11clif#4077
Closed

Uh oh!

Conversation

eli-schwartz commented Jul 17, 2022• edited by henryiii Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Suggested changelog entry:

Uh oh!

eli-schwartzJul 17, 2022

Choose a reason for hiding this comment

Uh oh!

Skylion007Jul 17, 2022

Choose a reason for hiding this comment

Uh oh!

eli-schwartzJul 17, 2022

Choose a reason for hiding this comment

Uh oh!

henryiiiJul 17, 2022

Choose a reason for hiding this comment

Uh oh!

henryiiiJul 17, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eli-schwartzJul 17, 2022

Choose a reason for hiding this comment

Uh oh!

eli-schwartzJul 17, 2022

Choose a reason for hiding this comment

Uh oh!

henryiii commented Jul 19, 2022

Uh oh!

eli-schwartz commented Jul 25, 2022

Uh oh!

eli-schwartz commented Jul 29, 2022

Uh oh!

henryiii commented Jul 29, 2022

Uh oh!

eli-schwartz commented Jul 29, 2022

Uh oh!

eli-schwartz commented Aug 1, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eli-schwartz commented Aug 5, 2022

Uh oh!

henryiii commented Aug 5, 2022

Uh oh!

eli-schwartz commented Aug 5, 2022

Uh oh!

rwgk commented Aug 8, 2022

Uh oh!

eli-schwartz commented Aug 8, 2022

Uh oh!

rwgk commented Aug 8, 2022

Uh oh!

eli-schwartz commented Aug 10, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

eli-schwartz commented Jul 17, 2022•
edited by henryiii
Loading

henryiiiJul 17, 2022•
edited
Loading

eli-schwartz commented Aug 1, 2022•
edited
Loading