`az --version` fails with SyntaxError on Python 3

[jaraco]

Description

az --version fails with SyntaxError on Python 3 due to Python 2 only library futures being installed.

Installing azure-cli 2.0.10 with pip on Python 3.6.1 produces this output. Importantly, these lines appear:

Collecting futures (from azure-multiapi-storage==0.1.0->azure-cli-vm->azure-cli)
  Downloading futures-3.1.1.tar.gz

Subsequently, operations in that environment that require pip will fail when compileall, which imports concurrent, is imported. For example:

$ PYTHONPATH=/opt/azure python -c "import pip"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/azure/pip/__init__.py", line 28, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/opt/azure/pip/vcs/subversion.py", line 9, in <module>
    from pip.index import Link
  File "/opt/azure/pip/index.py", line 31, in <module>
    from pip.wheel import Wheel, wheel_ext
  File "/opt/azure/pip/wheel.py", line 6, in <module>
    import compileall
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/compileall.py", line 20, in <module>
    from concurrent.futures import ProcessPoolExecutor
  File "/opt/azure/concurrent/futures/__init__.py", line 8, in <module>
    from concurrent.futures._base import (FIRST_COMPLETED,
  File "/opt/azure/concurrent/futures/_base.py", line 381
    raise exception_type, self._exception, self._traceback
                        ^
SyntaxError: invalid syntax

A similar traceback will appear with az --version when it imports pip.

Futures declares that it's Python 2 only, so shouldn't be installed on Python 3.

---

Environment summary

Install Method: How did you install the CLI? (e.g. pip, interactive script, apt-get, Docker, MSI, nightly)
Answer here: pip

CLI Version: What version of the CLI and modules are installed? (Use az --version)
Answer here: 2.0.10

OS Version: What OS and version are you using?
Answer here: macOS 10.12.5

Shell Type: What shell are you using? (e.g. bash, cmd.exe, Bash on Windows)
Answer here: zsh

[jaraco]

I was able to work around the issue by manually removing the concurrent package and then uninstalling futures.

$ rm -R /opt/azure/concurrent
$ PYTHONPATH=/opt/azure python -m pip uninstall -y futures
Uninstalling futures-3.1.1:
  Successfully uninstalled futures-3.1.1

[jaraco]

I see that azure-multiapi-storage does in fact declare the dependency conditionally. The problem is that the wheel that's produced will have the dependency fixed based on the environment in which the wheel is built.

I also observe that if the package is installed into a virtualenv (and not using PYTHONPATH) to make the azure libs available, the issue doesn't occur because the system concurrent takes precedence:

$ python -m venv azure
$ azure/bin/pip install -q azure-cli
$ azure/bin/python -m azure.cli --version
azure-cli (2.0.10)
...

Though I do encounter another issue if I try to invoke azure/bin/az.

[jaraco]

As I see it, there are a couple of ways to address the issue.

Environment markers

If you declare the dependencies with environment markers, pip will evaluate the dependencies late and not install futures on Python 3. Depending on which versions of setuptools and pip you want to support, the dependency looks like this:

setup(
    ...
    extras_require={
        ':python_version=="2.7"': ['futures'],
    },
)

or

setup(
    ...
    install_requires=[
        ...
        'futures; python_version=="2.7"',
    ],
)

with the latter syntax requiring newer versions of setuptools (19.x I think).

Separate wheels

If in the azure-multiapi-storage, you build the wheels separately for Python 2 and Python 3, the wheel for each target Python will have the proper dependency declaration.

[derekbekoe]

Jason R. Coombs (@jaraco) I've published an update to azure-multiapi-storage including this change (Azure/azure-multiapi-storage-python#5).

#4591 updates the dep. in Azure CLI. Once merged, it'll be fixed in the next version of the CLI.

[afelthe]

So how do we fix this issue in a Jupyter notebook?

[jiasli]

afelthe, Azure CLI has dropped support for Python 2 (#11363). Are you using the latest Azure CLI?