Allow PyYAML 6.x by freyes · Pull Request #214 · canonical/layer-basic

freyes · 2023-04-05T21:23:06Z

ops>=2.1.0 depends on PyYAML-6.x[0], allowing the installation of that version in layer-basic and the charm using layer-basic can lockdown on a more specific version of PyYAML.

It's important to keep reactive charms compatible with the ops framework, because there are compatibility layers to allow the use of interfaces in ops framework based charms[1]

[0] canonical/operator@9162c87
[1] openstack/charm-interface-hacluster@34b12aa

ops>=2.1.0 depends on PyYAML-6.x[0], allowing the installation of that version in layer-basic and the charm using layer-basic can lockdown on a more specific version of PyYAML. It's important to keep reactive charms compatible with the ops framework, because there are compatibility layers to allow the use of interfaces in ops framework based charms[1] [0] canonical/operator@9162c87 [1] openstack/charm-interface-hacluster@34b12aa

Python 3.6 is not available on github's ubuntu-latest image, the last image with python 3.6 was ubuntu-20.04, so switch to that version to run the linting with older versions of python on ubuntu-20.04

Error: ValueError: Error code '#' supplied to 'ignore' option does not match '^[A-Z]{1,3}[0-9]{0,3}$'

Replace whitelist_externals with allowlist_externals and use the full path to the commands.

freyes · 2023-04-06T13:41:15Z

Example of the failure that this patch aims to fix - https://review.opendev.org/c/openstack/charm-vault/+/879677

ajkavanagh

LGTM

nobuto-m · 2023-04-10T07:28:03Z


+# Cython is required to build PyYAML. To find out the supported versions check
+# https://github.com/cython/cython/issues/2800
+Cython


Not 100% sure because of this change, but a reactive charm has failed to build over the weekend. It's failing at Cython installation due to missing (Python.h: python3-dev).
Are you going to manage this dependency centrally or do you expect every single charm to have python3-dev?
2023-04-10 07:13:03.353319 | focal-medium | :: x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/root/snap/charmcraft/common/tmp8fu559sx/include -I/usr/include/python3.10 -c /root/snap/charmcraft/common/pip-install-s2qs0rbg/cython_6d98672645614f43bf70d0b7f222b29d/Cython/Plex/Scanners.c -o build/temp.linux-x86_64-cpython-310/root/snap/charmcraft/common/pip-install-s2qs0rbg/cython_6d98672645614f43bf70d0b7f222b29d/Cython/Plex/Scanners.o 2023-04-10 07:13:03.353337 | focal-medium | :: /root/snap/charmcraft/common/pip-install-s2qs0rbg/cython_6d98672645614f43bf70d0b7f222b29d/Cython/Plex/Scanners.c:21:10: fatal error: Python.h: No such file or directory 2023-04-10 07:13:03.353350 | focal-medium | :: 21 | #include "Python.h" 2023-04-10 07:13:03.353363 | focal-medium | :: | ^~~~~~~~~~ 2023-04-10 07:13:03.353375 | focal-medium | :: compilation terminated. 2023-04-10 07:13:03.353640 | focal-medium | :: error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1 2023-04-10 07:13:03.353668 | focal-medium | :: [end of output] 2023-04-10 07:13:03.353683 | focal-medium | :: 2023-04-10 07:13:03.353715 | focal-medium | :: note: This error originates from a subprocess, and is likely not a problem with pip. 2023-04-10 07:13:03.353737 | focal-medium | :: error: legacy-install-failure 2023-04-10 07:13:03.353748 | focal-medium | :: 2023-04-10 07:13:03.353759 | focal-medium | :: × Encountered error while trying to install package. 2023-04-10 07:13:03.353770 | focal-medium | :: ╰─> Cython

I think we'll need to make charm-tools grow a feature to allow layers express build-dependencies (and probably stage dependencies).

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I4c09f365c2ed7c7484c4e8fac87e5cdd6480bb10

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I2905e93c6154297a86eb41f2ae919f9bcdafabc9

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I0499d6c3c7dac72525cd93a9fd96e2ed9f6d62f3

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I8ac63e1a03d72c857ba04c0ec50b4491a00f7ef4

* Update charm-keystone-ldap from branch 'master' to 0f1520f4fa779f9ee4f8ecca071e37489852f3c0 - Fix build Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I8ac63e1a03d72c857ba04c0ec50b4491a00f7ef4

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I69895d604fb453cb6f1a48b5fa69544287fcd448

* Update charm-barbican-vault from branch 'master' to 9b95a69b8e6ca7ce159b7f5551719207979e788d - Fix build Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I69895d604fb453cb6f1a48b5fa69544287fcd448

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I172378e5bc8e919da4b21e75a5a9bcc19722b9ac

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I2905e93c6154297a86eb41f2ae919f9bcdafabc9 (cherry picked from commit f92e04d)

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I69895d604fb453cb6f1a48b5fa69544287fcd448 (cherry picked from commit 9b95a69)

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I8ac63e1a03d72c857ba04c0ec50b4491a00f7ef4 (cherry picked from commit 0f1520f)

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I4c09f365c2ed7c7484c4e8fac87e5cdd6480bb10 (cherry picked from commit 1ec2142)

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I0499d6c3c7dac72525cd93a9fd96e2ed9f6d62f3 (cherry picked from commit 24ed370)

Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I172378e5bc8e919da4b21e75a5a9bcc19722b9ac (cherry picked from commit c43146e)

libpython3-dev is required to build cython. See canonical/layer-basic#214 This problem is only evident now since we switched to charm 3.x in charmcraft builds in #115 ref. https://review.opendev.org/c/openstack/charm-keystone-ldap/+/880084

freyes added 4 commits April 5, 2023 17:17

Fix CI
6266cac
Python 3.6 is not available on github's ubuntu-latest image, the last image with python 3.6 was ubuntu-20.04, so switch to that version to run the linting with older versions of python on ubuntu-20.04

Fix tox.ini [flake8].ignore
9ac2329
Error: ValueError: Error code '#' supplied to 'ignore' option does not match '^[A-Z]{1,3}[0-9]{0,3}$'

Tox 4 compatibility fixes
dfd4417
Replace whitelist_externals with allowlist_externals and use the full path to the commands.

freyes force-pushed the bump-up-pyyaml branch from b1efedc to dfd4417Compare April 6, 2023 12:19

Add Cython to wheelhouse.txt
472b5e9

freyes force-pushed the bump-up-pyyaml branch from dc5eda1 to 472b5e9Compare April 6, 2023 13:18

freyes requested a review from ajkavanagh April 6, 2023 13:41

ajkavanagh approved these changes Apr 6, 2023
View reviewed changes

ajkavanagh merged commit a41b355 into canonical:masterApr 6, 2023

nobuto-m reviewed Apr 10, 2023
View reviewed changes

esunar mentioned this pull request Apr 10, 2023
python3-dev dependency issue #215
Open

openstack-mirroring pushed a commit to openstack/charm-manila-dashboard that referenced this pull request Apr 12, 2023
Fix build
1ec2142
Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I4c09f365c2ed7c7484c4e8fac87e5cdd6480bb10

openstack-mirroring pushed a commit to openstack/charm-cinder-netapp that referenced this pull request Apr 12, 2023
Fix build
24ed370
Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I0499d6c3c7dac72525cd93a9fd96e2ed9f6d62f3

openstack-mirroring pushed a commit to openstack/charm-keystone-ldap that referenced this pull request Apr 12, 2023
Fix build
0f1520f
Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I8ac63e1a03d72c857ba04c0ec50b4491a00f7ef4

openstack-mirroring pushed a commit to openstack/charm-barbican-vault that referenced this pull request Apr 12, 2023
Fix build
9b95a69
Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I69895d604fb453cb6f1a48b5fa69544287fcd448

openstack-mirroring pushed a commit to openstack/charm-keystone-kerberos that referenced this pull request Apr 12, 2023
Fix build
c43146e
Add libpython3-dev to the list of the build-packages, this is needed since canonical/layer-basic#214 . Change-Id: I172378e5bc8e919da4b21e75a5a9bcc19722b9ac

samuelallan72 mentioned this pull request Jul 25, 2024
Fix cython build in charmcraft build canonical/layer-filebeat#117
Merged

Navigation Menu

Uh oh!

Allow PyYAML 6.x - #214

Allow PyYAML 6.x#214
ajkavanagh merged 5 commits into
canonical:masterfrom
freyes:bump-up-pyyaml

freyes commented Apr 5, 2023

Uh oh!

freyes commented Apr 6, 2023

Uh oh!

ajkavanagh left a comment

Uh oh!

nobuto-mApr 10, 2023

Uh oh!

freyesApr 10, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

freyes commented Apr 5, 2023

Uh oh!

freyes commented Apr 6, 2023

Uh oh!

ajkavanagh left a comment

Choose a reason for hiding this comment

Uh oh!

nobuto-mApr 10, 2023

Choose a reason for hiding this comment

Uh oh!

freyesApr 10, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants