Unable to install on windows

[ompadu]

edit: Might be related to #8, missing binary wheels for python 3.9 and it tries to build it from source.

original post:
Ran the following command pip install bgfx-python==1.0.4 --user in the VS2019 command prompt with the output (replaced my actual user name with <user_name>):

Collecting bgfx-python==1.0.4
 Using cached bgfx-python-1.0.4.tar.gz (93 kB)
 Installing build dependencies ... done
 Getting requirements to build wheel ... done
 Preparing wheel metadata ... done
Collecting loguru
 Using cached loguru-0.5.3-py3-none-any.whl (57 kB)
Requirement already satisfied: colorama>=0.3.4 in c:\users\<user_name>\appdata\roaming\python\python39\site-packages (from loguru->bgfx-python==1.0.4) (0.4.4)
Collecting win32-setctime>=1.0.0
 Using cached win32_setctime-1.0.3-py3-none-any.whl (3.5 kB)
Building wheels for collected packages: bgfx-python
 Building wheel for bgfx-python (PEP 517) ... error
 ERROR: Command errored out with exit status 1:
 command: 'c:\users\<user_name>\scoop\apps\python\current\python.exe' 'C:\Users\<user_name>\AppData\Roaming\Python\Python39\site-packages\pip\_vendor\pep517\_in_process.py' build_wheel 'C:\Users\<user_name>\AppData\Local\Temp\tmp2tjh_ayy'
 cwd: C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5
 Complete output (152 lines):
 Not searching for unused variables given on the command line.
 -- The C compiler identification is MSVC 19.28.29335.0
 -- Detecting C compiler ABI info
 -- Detecting C compiler ABI info - done
 -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
 -- Detecting C compile features
 -- Detecting C compile features - done
 -- The CXX compiler identification is MSVC 19.28.29335.0
 CMake Warning (dev) at C:/Users/<user_name>/AppData/Local/Temp/pip-build-env-4lzdyvwp/overlay/Lib/site-packages/cmake/data/share/cmake-3.18/Modules/CMakeDetermineCXXCompiler.cmake:157 (if):
 Policy CMP0054 is not set: Only interpret if() arguments as variables or
 keywords when unquoted. Run "cmake --help-policy CMP0054" for policy
 details. Use the cmake_policy command to set the policy and suppress this
 warning.

 Quoted variables like "MSVC" will no longer be dereferenced when the policy
 is set to NEW. Since the policy is not set the OLD behavior will be used.
 Call Stack (most recent call first):
 CMakeLists.txt:4 (ENABLE_LANGUAGE)
 This warning is for project developers. Use -Wno-dev to suppress it.

 CMake Warning (dev) at C:/Users/<user_name>/AppData/Local/Temp/pip-build-env-4lzdyvwp/overlay/Lib/site-packages/cmake/data/share/cmake-3.18/Modules/CMakeDetermineCXXCompiler.cmake:177 (elseif):
 Policy CMP0054 is not set: Only interpret if() arguments as variables or
 keywords when unquoted. Run "cmake --help-policy CMP0054" for policy
 details. Use the cmake_policy command to set the policy and suppress this
 warning.

 Quoted variables like "MSVC" will no longer be dereferenced when the policy
 is set to NEW. Since the policy is not set the OLD behavior will be used.
 Call Stack (most recent call first):
 CMakeLists.txt:4 (ENABLE_LANGUAGE)
 This warning is for project developers. Use -Wno-dev to suppress it.

 -- Detecting CXX compiler ABI info
 -- Detecting CXX compiler ABI info - done
 -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
 -- Detecting CXX compile features
 -- Detecting CXX compile features - done
 -- Configuring done
 -- Generating done
 -- Build files have been written to: C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/_cmake_test_compile/build
 -- The C compiler identification is MSVC 19.28.29335.0
 -- The CXX compiler identification is MSVC 19.28.29335.0
 -- Detecting C compiler ABI info
 -- Detecting C compiler ABI info - done
 -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
 -- Detecting C compile features
 -- Detecting C compile features - done
 -- Detecting CXX compiler ABI info
 -- Detecting CXX compiler ABI info - done
 -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe - skipped
 -- Detecting CXX compile features
 -- Detecting CXX compile features - done
 -- Found Python3: C:/Users/<user_name>/scoop/apps/python/current/libs/python39.lib (found version "3.9.1") found components: Development Development.Module Development.Embed
 -- Found PythonInterp: c:/users/<user_name>/scoop/apps/python/current/python.exe (found version "3.9.1")
 -- Found PythonLibs: c:/users/<user_name>/scoop/apps/python/current/libs/python39.lib (found version "3.9.1")
 -- PYTHON_LIBRARIES = c:/users/<user_name>/scoop/apps/python/current/libs/python39.lib
 -- PYTHON_INCLUDE_DIRS = c:/users/<user_name>/scoop/apps/python/current/Include
 -- BX_DIR = C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/bx
 -- BIMG_DIR = C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/bimg
 -- BGFX_DIR = C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/bgfx
 -- PYBIND11_DIR = C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/pybind11
 -- FindWindowsSDK: Detected Visual Studio 2012 or newer, not using the _xp toolset variant: including SDK versions that drop XP support in search!
 -- Found WindowsSDK: C:/Program Files (x86)/Windows Kits/10
 'cmd' '/c' 'C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/scripts/rename_runtime.bat' 'C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/src/../dependencies/bgfx/.build/projects/vs2019'
 dir : Cannot find path 'C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c9
 5a5\dependencies\bgfx\.build\projects\' because it does not exist.
 At line:1 char:1
 + dir -Path C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx- ...
 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo : ObjectNotFound: (C:\Users\<user_name>\A...build\projects\:String) [Get-ChildItem], ItemNotFound
 Exception
 + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand

 Microsoft (R) Build Engine version 16.8.2+25e4d540b for .NET Framework
 Copyright (C) Microsoft Corporation. All rights reserved.

 MSBUILD : error MSB1009: Project file does not exist.
 Switch: bgfx.sln
 -- Performing Test Weak Link MODULE -> SHARED (gnu_ld_ignore) - Failed
 -- Performing Test Weak Link MODULE -> SHARED (osx_dynamic_lookup) - Failed
 -- Performing Test Weak Link MODULE -> SHARED (no_flag) - Failed
 _modinit_prefix:PyInit_
 _modinit_prefix:PyInit_
 -- Configuring done
 CMake Error at CMakeLists.txt:79 (add_library):
 Cannot find source file:

 C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/dependencies/bgfx/3rdparty/dear-imgui/imgui.cpp

 Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
 .hpp .hxx .in .txx


 CMake Error at CMakeLists.txt:93 (add_library):
 Cannot find source file:

 C:/Users/<user_name>/AppData/Local/Temp/pip-install-_fsm2ab_/bgfx-python_c762d88519934ecdbe890d32471c95a5/dependencies/bgfx/tools/shaderc/shaderc_glsl.cpp

 Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
 .hpp .hxx .in .txx


 CMake Error at CMakeLists.txt:79 (add_library):
 No SOURCES given to target: bgfx_lib


 CMake Error at CMakeLists.txt:93 (add_library):
 No SOURCES given to target: shaderc


 CMake Generate step failed. Build files cannot be regenerated correctly.
 File "C:\Users\<user_name>\AppData\Local\Temp\pip-build-env-4lzdyvwp\overlay\Lib\site-packages\skbuild\setuptools_wrap.py", line 582, in setup
 env = cmkr.configure(cmake_args,
 File "C:\Users\<user_name>\AppData\Local\Temp\pip-build-env-4lzdyvwp\overlay\Lib\site-packages\skbuild\cmaker.py", line 229, in configure
 raise SKBuildError(


 --------------------------------------------------------------------------------
 -- Trying "Ninja (Visual Studio 15 2017 Win64 v141)" generator
 --------------------------------
 ---------------------------
 ----------------------
 -----------------
 ------------
 -------
 --
 --
 -------
 ------------
 -----------------
 ----------------------
 ---------------------------
 --------------------------------
 -- Trying "Ninja (Visual Studio 15 2017 Win64 v141)" generator - success
 --------------------------------------------------------------------------------

 Configuring Project
 Working directory:
 C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\_skbuild\win-amd64-3.9\cmake-build
 Command:
 cmake 'C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\src' -G Ninja '-DCMAKE_INSTALL_PREFIX:PATH=C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\_skbuild\win-amd64-3.9\cmake-install' '-DPYTHON_EXECUTABLE:FILEPATH=c:\users\<user_name>\scoop\apps\python\current\python.exe' -DPYTHON_VERSION_STRING:STRING=3.9.1 '-DPYTHON_INCLUDE_DIR:PATH=c:\users\<user_name>\scoop\apps\python\current\Include' '-DPYTHON_LIBRARY:FILEPATH=c:\users\<user_name>\scoop\apps\python\current\libs\python39.lib' -DSKBUILD:BOOL=TRUE '-DCMAKE_MODULE_PATH:PATH=C:\Users\<user_name>\AppData\Local\Temp\pip-build-env-4lzdyvwp\overlay\Lib\site-packages\skbuild\resources\cmake' -DCMAKE_BUILD_TYPE:STRING=Release

 Traceback (most recent call last):

 An error occurred while configuring with CMake.
 Command:
 cmake 'C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\src' -G Ninja '-DCMAKE_INSTALL_PREFIX:PATH=C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\_skbuild\win-amd64-3.9\cmake-install' '-DPYTHON_EXECUTABLE:FILEPATH=c:\users\<user_name>\scoop\apps\python\current\python.exe' -DPYTHON_VERSION_STRING:STRING=3.9.1 '-DPYTHON_INCLUDE_DIR:PATH=c:\users\<user_name>\scoop\apps\python\current\Include' '-DPYTHON_LIBRARY:FILEPATH=c:\users\<user_name>\scoop\apps\python\current\libs\python39.lib' -DSKBUILD:BOOL=TRUE '-DCMAKE_MODULE_PATH:PATH=C:\Users\<user_name>\AppData\Local\Temp\pip-build-env-4lzdyvwp\overlay\Lib\site-packages\skbuild\resources\cmake' -DCMAKE_BUILD_TYPE:STRING=Release
 Source directory:
 C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\src
 Working directory:
 C:\Users\<user_name>\AppData\Local\Temp\pip-install-_fsm2ab_\bgfx-python_c762d88519934ecdbe890d32471c95a5\_skbuild\win-amd64-3.9\cmake-build
 Please see CMake's output for more information.
 ----------------------------------------
 ERROR: Failed building wheel for bgfx-python
Failed to build bgfx-python
ERROR: Could not build wheels for bgfx-python which use PEP 517 and cannot be installed directly

[fbertola]

Hello @ompadu ! Yes I faced a major roadblock with pypi not accepting the wheels due to a file size limit. I requested a limit increase but so far no response. In the meantime, try building from source cloning the repository and its submodules.

I'm in the process to rewrite a major portion of the c++ bindings with cppyy. When its ready I'll try to release wheels for Python 3.9 and deprecate 3.6.

Thanks for your patience.

[ompadu]

Thank you for the answer. No problem, I am in no big rush,

[JackWitherell]

Hate to bump this when it's known that something already needs to be fixed, but I wanted to mention that it's not possible to build the repo and it's modules using the readme. Hoping to help someone or possibly get someone up to my current block to look further into it.

Unclear 1

The readme suggests to run pip setup.py install which isn't correct at all, the correct statement should be python setup.py install. This is because setup.py can't be used as a parameter for pip.

Building Dependency 1

On attempting to install, however, I get an error in setup.py stating that there's no module named skbuild. scikit-build is a dependency on setup.py and isn't installed by default.

Unclear 2

After attempting this however, I get issues with skbuild, which states that there's a problem with cmake on my machine. I installed cmake from chocolatey, which wasn't directly suggested in the readme but is an easy mistake to make as the readme states to use chocolatey to install an out of date version of visual studio.

it should be made clear in the readme that the versions of cmake, make and ninja that are installed should be installed through pip, not chocolatey.

Unclear 3

After correcting the issue however, scikit-build makes it clear that there's no generator - hence the readme stating to install visual studio 2017 and to call vcvarsall.bat. However, after installing vs2017 community with chocolatey and attempting to call the vc vars setup script, it's suggested that it doesn't exist. Looking into it, vcvarsall.bat never existed in vs2017 and is a vs2015 only feature. The suggestion for python users is to... use a wheel instead.

In lieu of a functioning wheel, I started trying to find the alternative command for this, which is call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools\VsDevCmd.bat" -arch=x64 to set up these vars.

This functions properly... until I try building the repo.

Building Dependency 2

Attempting this causes tons of CMake errors due to not having the Windows 8.1 SDK. Yikes. Not a good look.

Building Dependency 3

After installing the Windows 8.1 SDK through the visual studio installer... LibClang can't be found. This is apparently needed to build cppyy.
Instead of trying to figure out what wonky dev setup allows LibClang to be pulled out of thin air, I ended up running pip install cppyy to intervene.

Misconfiguration?

After that, I got an error from ninja stating that dependencies/bgfx/.build/win64_vs2017/bin/bimgRelease.lib is needed.
The dependencies tree while building doesn't include this but includes one for win32 instead.
Either this is a misconfiguration in the CMakeLists file or further clueing is needed to get the 64 bit version of bimgRelease.lib to compile.
My fix for this was to change the version that is actually used by cmake in cmakelists.txt (changed this to win32) as I don't know how to force the win64 version to build instead. I think this is due to the 64 bit version being the only one supported for a while, with 32 bit releasing later and becoming the default.

bgfx-python/src/CMakeLists.txt

Line 38 in f83c43c

set(BGFX_BUILD_DIR win64_vs2017)

Blocked

And lastly, this is where I hit a hard wall.

Ninja started failing due to an improperly formatted command generated by FindCppyy.cmake. It's very clear that this cmake file was never meant to be used in its current state, especially since it's located in the experimental folder of the official ROOT repo for Python's rootcling implementation: https://github.com/eric-erki/The-official-repository-for-ROOT/blob/7e47c421a88fdd75b7dce058ece9b4f31135b41b/bindings/pyroot_experimental/cppyy/cppyy-backend/cling/python/cppyy_backend/cmake/FindCppyy.cmake

TL;DR

There's a lot more prerequisites to building, most of which have come out morphing visual studio distributions and bad recommendations on behalf of building this from source. As the only other set of recommended python bindings for BGFX are also practically non-functional, we're left with kinda a hot mess of no real python support in bgfx.

I'd be interested to know where we're at, if anywhere on the proposed rewrite. I think it's a really bad look for BGFX to have this listed on their page and have it be practically nonfunctional.