Support Python 3.12

[chenrui333]

Description

Python 3.12 has been released on 2023-10-02: https://www.python.org/downloads/release/python-3120/

Azure CLI needs to support Python 3.12 like how we supported 3.10 (#19857), 3.11 (#24494).

TODOs

• Knack supports Python 3.12
  • Mark Python 3.12 support microsoft/knack#279
• azdev supports Python 3.12
  • Bump pylint to 3 azure-cli-dev-tools#458
• Dependencies are updated to the Python 3.12 compatible version
  • pymsalruntime (dependency of msal[broker]) {Core} Bump MSAL to 1.27.0 and pin pymsalruntime to 0.14.0 #28556
  • {Packaging} Bump cffi to 1.16.0 #29466
  • {Packaging} Bump PyWin32 to 306 #29486
• CI passes with Python 3.12
  • Remove distutils {Core} Remove deprecated usages of distutils to support Python 3.12 #28796
  • {Packaging} Add setuptools to dependency #27196
  • {Misc.} Remove deprecated assertRaisesRegexp #29504
  • {Misc.} Remove deprecated assertEquals #29515
  • {AKS} Fix enum check in test #29517
  • Remove assertDictContainsSubset and assertNotEquals Please port to Python3.12 #28194
    • {Misc.} Remove all occurrences of assertNotEquals #29120
    • {Misc.} Remove all occurrences of assertDictContainsSubset #29121
  • Hundreds of "SyntaxWarning: invalid escape sequence" will break in future python versions #28829
  • {Misc.} Improve code style and prepare for pylint 3 #29373
  • [Packaging] Support Python 3.12 #29465
  • [Packaging] Bump embedded Python version to 3.12.7 #29887
  • {CI} Replace deprecated assertRaisesRegexp and assertEquals azure-cli-extensions#8010
  • {CI} Update pipeline to use Python 3.12 azure-cli-extensions#8116

[azure-client-tools-bot-prd]

Hi Rui Chen (@chenrui333)
Find similar issue #19857.

Issue title	Support Python 3.10
Create time	2021-10-12
Comment number	6

Possible solution:
As we did for Python 3.10 and 3.11, Azure CLI needs to support Python 3.12. We can follow the same steps we did for Python 3.10 support, which includes updating dependencies like pytest, pylint, and ensuring all test cases pass. Additionally, we need to make sure that Knack and azdev also support Python 3.12. We can refer to the pull requests for Knack and azdev that added support for Python 3.10 and update them accordingly for Python 3.12. Once we have updated the dependencies and ensured that all test cases pass, we can release a new version of Azure CLI that supports Python 3.12.

---

Please confirm if this resolves your issue.

[yonzhan]

Thank you for opening this issue, we will look into it.

[chenrui333]

I find there is no open issue to support py3.12, create this one for the team.

[Gyarbij]

Any progress on this and will this impact App Service Python 3.12 availability?

[borjamunozf]

Could be this the explanation why installing azure-cli with Python 3.12 throw this error complining about pymsalruntime?

building 'pymsalruntime.pymsalruntime' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pymsalruntime
Successfully built antlr4-python3-runtime
Failed to build pymsalruntime
ERROR: Could not build wheels for pymsalruntime, which is required to install pyproject.toml-based projects

[spauka]

Indeed it seems like the dependency of pymsalruntime means that azure-cli is not compatible with Python 3.12. The dependancy chain seems to be:
pymsalruntime<0.14,>=0.13.2 (from msal[broker]==1.24.0b2->azure-cli-core==2.53.1->azure-cli->az-cli)

The pymsalruntime library seems to use the deprecated and removed functions PyUnicode_AsUnicode and PyUnicode_FromUnicode, which means that the library cannot be built on the latest python.

[mikeeq]

Fedora 39 was just released, and it's containing Python 3.12. It is a huge blocker now for all Fedora users, as it's not possible to run Azure CLI "as is" without any additional hassle (running it in a container or with older version of python).

❯ az
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/__main__.py", line 39, in <module>
    az_cli = get_default_cli()
             ^^^^^^^^^^^^^^^^^
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/core/__init__.py", line 917, in get_default_cli
    from azure.cli.core.azlogging import AzCliLogging
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/core/azlogging.py", line 30, in <module>
    from azure.cli.core.commands.events import EVENT_INVOKER_PRE_CMD_TBL_TRUNCATE
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 25, in <module>
    from azure.cli.core.extension import get_extension
  File "/opt/virtualenv/azure-cli/lib/python3.12/site-packages/azure/cli/core/extension/__init__.py", line 11, in <module>
    from distutils.sysconfig import get_python_lib  # pylint: disable=deprecated-module
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'distutils'
❯ uname -a
Linux 6.5.10-300.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Nov  2 20:01:06 UTC 2023 x86_64 GNU/Linux
❯ python --version
Python 3.12.0

Links:

• https://fedoraproject.org/wiki/Changes/Python3.12
• https://peps.python.org/pep-0632/