Support Python 3.11

[jiasli]

Description

Python 3.11 has been released on 2022-10-24: https://www.python.org/downloads/release/python-3110/

Azure CLI needs to support Python 3.11 like how we supported 3.9 (#16611), 3.10 (#19857).

TODOs

• Knack supports Python 3.11
  • Declare support for Python 3.11 and drop support for Python 3.7 microsoft/knack#275
• azdev supports Python 3.11
  • pylint (Bump Pylint to 2.17 and flake8 to 6.0 azure-cli-dev-tools#359)
  • vcrpy (same as above)
• Dependencies are updated to the Python 3.11 compatible version
  • psycopg2 ({Packaging} Update psycopg2 to support Python 3.11 #24671)
  • pywin32 ({Packaging} Bump PyWin32 to 305 #24982)
  • cffi psutil ({Packaging} Bump cffi to 1.15.1 and psutil to 5.9.5 #26842)
• CI passes with Python 3.11
  • {Core} Make parser compatible with Python 3.11's argparse #24497
  • {Misc.} Improve code style and prepare for pylint 2.17 #26685
  • {CI} Make .flake8 compatible with Flake8 6.0.0 #25370
  • [Packaging] Support Python 3.11 #26923
    • {Compute} Change KeyVaultClient to CertificateClient in vm module #26991
    • {Keyvault} Vendor azure-keyvault track1 SDK to make customization for supporting Python 3.11 #27148
    • {aaz} Rename aaz profile package name by adding profile_ prefix for azure stack profiles. #27073
  • [Packaging] Bump embedded Python version to 3.11.5 #26749

[jiasli]

psycopg2 (introduced by #22459) fails to install on Python 3.11:

Collecting psycopg2==2.9.3
  Downloading psycopg2-2.9.3.tar.gz (380 kB)
     ------------------------------------- 380.6/380.6 kB 23.1 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error

  python setup.py egg_info did not run successfully.
  exit code: 1

  [23 lines of output]
  running egg_info
  creating C:\Users\username\AppData\Local\Temp\pip-pip-egg-info-4nrehwws\psycopg2.egg-info
  writing C:\Users\username\AppData\Local\Temp\pip-pip-egg-info-4nrehwws\psycopg2.egg-info\PKG-INFO
  writing dependency_links to C:\Users\username\AppData\Local\Temp\pip-pip-egg-info-4nrehwws\psycopg2.egg-info\dependency_links.txt
  writing top-level names to C:\Users\username\AppData\Local\Temp\pip-pip-egg-info-4nrehwws\psycopg2.egg-info\top_level.txt
  writing manifest file 'C:\Users\username\AppData\Local\Temp\pip-pip-egg-info-4nrehwws\psycopg2.egg-info\SOURCES.txt'

  Error: pg_config executable not found.

  pg_config is required to build psycopg2 from source.  Please add the directory
  containing pg_config to the $PATH or specify the full executable path with the
  option:

      python setup.py build_ext --pg-config /path/to/pg_config build ...

  or with the pg_config option in 'setup.cfg'.

  If you prefer to avoid building psycopg2 from source, please install the PyPI
  'psycopg2-binary' package instead.

  For further information please check the 'doc/src/install.rst' file (also at
  <https://www.psycopg.org/docs/install.html>).

  [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Encountered error while generating package metadata.

[bebound]

I guess we need to update psycopg2 to 2.9.5
psycopg/psycopg2#1514

[bebound]

azdev's dependency pylint==2.11.1 breaks vcrpy, as wrapt 1.13 does not support 3.11.
wrapt support 3.11 in 1.14 https://wrapt.readthedocs.io/en/latest/changes.html#version-1-14-0
pylint will support 3.11 in 2.16

Version 1.14.0
Bugs Fixed
Python 3.11 dropped inspect.formatargspec() which was used in creating signature changing decorators. Now bundling a version of this function which uses Parameter and Signature from inspect module when available. The replacement function is exposed as wrapt.formatargspec() if need it for your own code.

├── pylint==2.11.1                                                                                  python code static checker
│   ├── astroid<2.9,>=2.8.0                                                                         An abstract syntax tree for Python with inference support.
│   │   ├── lazy-object-proxy>=1.4.0                                                                A fast and thorough lazy object proxy.
│   │   ├── setuptools>=20.0                                                                        Easily download, build, install, upgrade, and uninstall Python packages
│   │   └── wrapt<1.14,>=1.11

ImportError while importing test module '/mnt/vss/_work/1/s/src/azure-cli/azure/cli/command_modules/network/tests/latest/test_private_endpoint_commands.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
src/azure-cli/azure/cli/command_modules/network/tests/latest/test_private_endpoint_commands.py:13: in <module>
    from azure.cli.testsdk import (
src/azure-cli-testsdk/azure/cli/testsdk/__init__.py:6: in <module>
    from .scenario_tests import live_only, record_only, get_sha1_hash
src/azure-cli-testsdk/azure/cli/testsdk/scenario_tests/__init__.py:13: in <module>
    from .base import IntegrationTestBase, ReplayableTest, LiveTest
src/azure-cli-testsdk/azure/cli/testsdk/scenario_tests/base.py:14: in <module>
    import vcr
env/lib/python3.11/site-packages/vcr/__init__.py:2: in <module>
    from .config import VCR
env/lib/python3.11/site-packages/vcr/config.py:11: in <module>
    from .cassette import Cassette
env/lib/python3.11/site-packages/vcr/cassette.py:8: in <module>
    import wrapt
env/lib/python3.11/site-packages/wrapt/__init__.py:10: in <module>
    from .decorators import (adapter_factory, AdapterFactory, decorator,
env/lib/python3.11/site-packages/wrapt/decorators.py:34: in <module>
    from inspect import ismethod, isclass, formatargspec
E   ImportError: cannot import name 'formatargspec' from 'inspect' (/opt/hostedtoolcache/Python/3.11.0/x64/lib/python3.11/inspect.py)

[bebound]

vcrpy supports 3.11 in kevin1024/vcrpy#667, but it's dependency multidict does not provide wheels for 3.11. Here is the issue Wheels for CPython 3.11.

vcrpy                   Automatically mock your HTTP interactions to simplify and speed up testing
├── PyYAML              YAML parser and emitter for Python
├── six>=1.5            Python 2 and 3 compatibility utilities
├── wrapt               Module for decorators, wrappers and monkey patching.
└── yarl                Yet another URL library
    ├── idna>=2.0       Internationalized Domain Names in Applications (IDNA)
    └── multidict>=4.0  multidict implementation

PS: This dependency tree is generated with johnnydep

[mrmuskrat]

Is there a workaround for users on an operating system with python3.11 preinstalled (like Fedora 37)?

[mrmuskrat]

Is there a workaround for users on an operating system with python3.11 preinstalled (like Fedora 37)?

I did find a workaround for myself this afternoon.

Steps:

1. Remove the azure-cli RPM: sudo dnf remove azure-cli
2. Install Python 3.10: sudo dnf install python3.10
3. Create a virtual environment for Python: python3.10 -m venv ~/.venv
4. Activate the virtual environment: . ~/.venv/bin/activate
5. Install the Azure CLI using pip: pip install azure-cli