Skip to content

fix: Add python include for compilation - #803

Closed
illicitonion wants to merge 2 commits into
bazel-contrib:mainfrom
illicitonion:fix-compile-cc-on-m1
Closed

fix: Add python include for compilation#803
illicitonion wants to merge 2 commits into
bazel-contrib:mainfrom
illicitonion:fix-compile-cc-on-m1

Conversation

@illicitonion

Copy link
Copy Markdown
Contributor

PR Checklist

Please check if your PR fulfills the following requirements:

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

PR Type

What kind of change does this PR introduce?

  • Bugfix

What is the current behavior?

When compiling pypi wheels, the rules do not set the correct includes on darwin-arm64. In our case we tried using the psutil package. And got the following error in this repro:

$ bazel build :main[...]Failed to build psutil ( error: subprocess-exited-with-error × Building wheel for psutil (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [50 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.macosx-11.0-arm64-cpython-310 creating build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/_pswindows.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/_common.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/_psosx.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/_psbsd.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/_psaix.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/_pslinux.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/_compat.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/_psposix.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil copying psutil/_pssunos.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil creating build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_contracts.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_connections.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/runner.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_unicode.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_misc.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_posix.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_linux.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_sunos.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/__init__.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_aix.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_process.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_bsd.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_system.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_osx.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_memleaks.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_windows.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/__main__.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/foo.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests copying psutil/tests/test_testutils.py -> build/lib.macosx-11.0-arm64-cpython-310/psutil/tests running build_ext building 'psutil._psutil_osx' extension creating build/temp.macosx-11.0-arm64-cpython-310 creating build/temp.macosx-11.0-arm64-cpython-310/psutil creating build/temp.macosx-11.0-arm64-cpython-310/psutil/arch creating build/temp.macosx-11.0-arm64-cpython-310/psutil/arch/osx Compiling with an SDK that doesn't seem to exist: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk Please check your Xcode installation clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -mmacosx-version-min=11.0 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -fPIC -I/var/folders/nx/g31mzz4d2mx2t4fsl1dwhycc0000gn/T/tmps98q1tpw/tools/deps/include -I/var/folders/nx/g31mzz4d2mx2t4fsl1dwhycc0000gn/T/tmps98q1tpw/tools/deps/include/ncursesw -I/var/folders/nx/g31mzz4d2mx2t4fsl1dwhycc0000gn/T/tmps98q1tpw/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -isysroot /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=591 -DPSUTIL_OSX=1 -I/install/include/python3.10 -c psutil/_psutil_common.c -o build/temp.macosx-11.0-arm64-cpython-310/psutil/_psutil_common.o psutil/_psutil_common.c:9:10: fatal error: 'Python.h' file not found #include <Python.h> ^~~~~~~~~~ 1 error generated. error: command '/usr/bin/clang' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for psutilERROR: Failed to build one or more wheelsTraceback (most recent call last): File "/private/var/tmp/_bazel_xxx/yyy/external/python3_10_aarch64-apple-darwin/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/private/var/tmp/_bazel_xxx/yyy/external/python3_10_aarch64-apple-darwin/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/private/var/tmp/_bazel_xxx/yyy/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 81, in <module> main() File "/private/var/tmp/_bazel_xxx/yyy/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 57, in main subprocess.run(pip_args, check=True, env=env) File "/private/var/tmp/_bazel_xxx/yyy/external/python3_10_aarch64-apple-darwin/lib/python3.10/subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args,subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_xxx/yyy/external/python3_10_aarch64-apple-darwin/bin/python3', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '--index=https://pypi.apple.com/simple', '--index-url', 'https://pypi.apple.com/simple', '--index-url', 'https://pypi.apple.com/simple', '-r', '/var/folders/0l/nfqx5j4j08dg3s3nwx51vqjw0000gn/T/tmpslxj3627']' returned non-zero exit status 1.)

What is the new behavior?

Adds the correct import path to be able to find Python.h

Does this PR introduce a breaking change?

  • Yes
  • No

@groodt

Copy link
Copy Markdown
Contributor

Thanks @illicitonion !

Your fix was inspiration for a bigger fix and included here:
#809

@groodtgroodt closed this Aug 27, 2022
@illicitonion
illicitonion deleted the fix-compile-cc-on-m1 branch August 29, 2022 12:53
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@illicitonion@groodt