Failure on SLES 15 SP3 VM: No module named azure.cli.__main__

[jiasli]

Describe the bug

Following https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=zypper, a freshly installed Azure CLI fails with

> az -v
/usr/bin/python3.6: No module named azure.cli.__main__; 'azure.cli' is a package and cannot be directly executed

This is because SLES (SUSE Linux Enterprise Server) 15 SP3 VM has a pre-installed Azure CLI 2.16.0 which is outdated and conflicts with latest version.

> ls -d /usr/lib/python3.6/site-packages/azure_cli*
/usr/lib/python3.6/site-packages/azure_cli-2.16.0-py3.6.egg-info
/usr/lib/python3.6/site-packages/azure_cli_command_modules_nspkg-2.0.3-py3.6.egg-info
/usr/lib/python3.6/site-packages/azure_cli_core-2.16.0-py3.6.egg-info
/usr/lib/python3.6/site-packages/azure_cli_nspkg-3.0.4-py3.6.egg-info
/usr/lib/python3.6/site-packages/azure_cli_telemetry-1.0.6-py3.6.egg-info

> ls -l /usr/lib/python3.6/site-packages/azure/cli
total 16
-rw-r--r--  1 root root    0 Jun 28  2019 __init__.py
-rw-r--r--  1 root root 5532 Dec  4  2020 __main__.py
drwxr-xr-x  2 root root   37 Dec 16 11:27 __pycache__
drwxr-xr-x 65 root root 4096 Dec 16 11:28 command_modules
drwxr-xr-x  8 root root 4096 Dec 16 11:28 core
drwxr-xr-x  4 root root   93 Dec 16 11:27 telemetry

After installing the latest version, /usr/lib/python3.6/site-packages/azure/cli/__main__.py is removed

> ls -l /usr/lib/python3.6/site-packages/azure/cli
total 4
-rw-r--r-- 1 root root    0 Jun 28  2019 __init__.py
drwxr-xr-x 2 root root   37 Dec 16 11:27 __pycache__
drwxr-xr-x 3 root root   44 Dec 27 08:58 command_modules
drwxr-xr-x 8 root root 4096 Dec 16 11:28 core
drwxr-xr-x 4 root root   93 Dec 16 11:27 telemetry

leaving /usr/lib/python3.6/site-packages/azure/cli a regular package, conflicting with the latest Azure CLI which is a namespace package under /usr/lib64/az/lib/python3.6/site-packages/azure/cli (#14372).

Solution

Remove the pre-installed Azure CLI before installing the latest one:

> sudo zypper rm -y --clean-deps azure-cli

If you have already installed the latest one, revert back to the pre-installed one first and reinstall the latest one:

> sudo zypper install --oldpackage azure-cli-2.16.0-6.7.1
> sudo zypper rm -y --clean-deps azure-cli

The package name may vary on different system version, run zypper --no-refresh info azure-cli on a new SLES 15 SP3 VM to check the source package format:

> zypper --no-refresh info azure-cli

Information for package azure-cli:
----------------------------------
Repository     : @System
Name           : azure-cli
Version        : 2.16.0-6.7.1
Arch           : noarch
Vendor         : SUSE LLC <https://www.suse.com/>
Support Level  : unknown
Installed Size : 15.9 MiB
Installed      : Yes
Status         : up-to-date
Source package : azure-cli-2.16.0-6.7.1.src
Summary        : Microsoft Azure CLI 2.0
Description    :
    Microsoft Azure CLI 2.0 Command Line Utilities

[jiasli]

John Paul Adrian Glaubitz (@glaubitz), as we are continuously seeing issues with the pre-installed azure-cli, do you think if it is possible to remove it from SUSE images? This way, our customers can always get the latest version distributed officially by Microsoft.

[glaubitz]

Jiashuo Li (@jiasli) No, this is not possible as we must provide our own distribution of the Azure packages as we cannot provide support to customers which install the packages from upstream.

Customers are advised to report issues with the Azure packages in openSUSE/SUSE with SUSE's Bugzilla database.

If you have customers reporting SUSE-specific issues here, please close the bug report as invalid and refer them to our bug database.

[glaubitz]

FWIW, it looks like the comment from the other bug report you are referring to talks about installing packages from Microsoft's own RPM repositories.

So, the bug here is that Microsoft's own RPM packages are missing the necessary Conflicts field in their RPM spec file so that installation of Microsoft's own RPM packages will cause the package manager to remove SUSE's Azure packages from the system.

Also, according to the naming scheme of Microsoft's own azure-cli packages, these packages are built for RHEL7 ("el7") [1], so I'm not sure why Microsoft recommends installing these packages on openSUSE/SLE in the first place.

[1] https://packages.microsoft.com/yumrepos/azure-cli/

[jiasli]

Thanks John Paul Adrian Glaubitz (@glaubitz), we really appreciate your contribution to make Azure CLI available on openSUSE.

Customers are advised to report issues with the Azure packages in openSUSE/SUSE with SUSE's Bugzilla database.

Do you mind letting us know the official link or instruction for reporting Azure CLI issues to SUSE's Bugzilla database?

So, the bug here is that Microsoft's own RPM packages are missing the necessary Conflicts field in their RPM spec file so that installation of Microsoft's own RPM packages will cause the package manager to remove SUSE's Azure packages from the system.

Could you provide more information on how we could leverage the Conflicts field? Actually, azure-cli and azure* packages are released at very rapid pace. We can't exhaust all possible combinations and figure out which combination is compatible or not.

Also, according to the naming scheme of Microsoft's own azure-cli packages, these packages are built for RHEL7 ("el7") [1], so I'm not sure why Microsoft recommends installing these packages on openSUSE/SLE in the first place.

This is simply because the RPM packages works universally, right? We haven't seen problems with the RPM package on other Linux distribution's yet. At least, the el7 one works perfectly on openSUSE.

Still, I think openSUSE can provide an official instruction on removing the pre-installed azure-cli package if customer wants to use the one distributed by us (Microsoft) at https://packages.microsoft.com/yumrepos/azure-cli/.

[glaubitz]

Thanks John Paul Adrian Glaubitz (@glaubitz), we really appreciate your contribution to make Azure CLI available on openSUSE.

Customers are advised to report issues with the Azure packages in openSUSE/SUSE with SUSE's Bugzilla database.

Do you mind letting us know the official link or instruction for reporting Azure CLI issues to SUSE's Bugzilla database?

Use https://bugzilla.suse.com and report against the the Cloud:Tools component after choosing openSUSE Leap as the distribution.

So, the bug here is that Microsoft's own RPM packages are missing the necessary Conflicts field in their RPM spec file so that installation of Microsoft's own RPM packages will cause the package manager to remove SUSE's Azure packages from the system.

Could you provide more information on how we could leverage the Conflicts field? Actually, azure-cli and azure* packages are released at very rapid pace. We can't exhaust all possible combinations and figure out which combination is compatible or not.

It looks like you are combining azure-cli-core and azure-cli into one RPM while we have separate packages for that in openSUSE. So, you should add azure-cli-core to the Conflicts field of your RPM spec file.

Also, according to the naming scheme of Microsoft's own azure-cli packages, these packages are built for RHEL7 ("el7") [1], so I'm not sure why Microsoft recommends installing these packages on openSUSE/SLE in the first place.

This is simply because the RPM packages works universally, right? We haven't seen problems with the RPM package on other Linux distribution's yet. At least, the el7 one works perfectly on openSUSE.

There is no guarantee that it works though. You get this guarantee only if you build the packages on the distribution where you want to ship them.

Still, I think openSUSE can provide an official instruction on removing the pre-installed azure-cli package if customer wants to use the one distributed by us (Microsoft) at https://packages.microsoft.com/yumrepos/azure-cli/.

There is no need for these instructions if you just add the proper Conflicts. This way, the installation of your packages will automatically cause the openSUSE packages to be removed.

[jiasli]

There is no need for these instructions if you just add the proper Conflicts. This way, the installation of your packages will automatically cause the openSUSE packages to be removed.

This is actually not that easy. Adding azure-cli-core to Conflict to RPM and then installing our RPM will only remove azure-cli-core, right? Other dependencies like azure-mgmt-* can also causes all kinds of issues. It's difficult for us to enumerate all dependencies in Conflict.

[glaubitz]

There is no need for these instructions if you just add the proper Conflicts. This way, the installation of your packages will automatically cause the openSUSE packages to be removed.

This is actually not that easy. Adding azure-cli-core to Conflict to RPM and then installing our RPM will only remove azure-cli-core, right? Other dependencies like azure-mgmt-* can also causes all kinds of issues. It's difficult for us to enumerate all dependencies in Conflict.

Hmm, I wasn't aware you have included the Azure SDK in your azure-cli package as well.

In this case, I suggest adding python-azure-core, python-azure-common and python-azure-mgmt-core to Conflicts. This should remove most of the SDK packages.

[jiasli]

We have so many dependencies:

azure-cli/src/azure-cli/setup.py

Lines 66 to 132 in d04b359

	'azure-mgmt-advisor==9.0.0',
	'azure-mgmt-apimanagement~=0.2.0',
	'azure-mgmt-appconfiguration~=2.0.0',
	'azure-mgmt-applicationinsights~=1.0.0',
	'azure-mgmt-authorization~=0.61.0',
	'azure-mgmt-batchai~=7.0.0b1',
	'azure-mgmt-batch~=16.0.0',
	'azure-mgmt-billing==6.0.0',
	'azure-mgmt-botservice~=0.3.0',
	'azure-mgmt-cdn==11.0.0',
	'azure-mgmt-cognitiveservices~=13.0.0',
	'azure-mgmt-compute~=23.1.0',
	'azure-mgmt-consumption~=2.0',
	'azure-mgmt-containerinstance~=9.1.0',
	'azure-mgmt-containerregistry==8.2.0',
	'azure-mgmt-containerservice~=16.1.0',
	'azure-mgmt-cosmosdb~=7.0.0b2',
	'azure-mgmt-databoxedge~=1.0.0',
	'azure-mgmt-datalake-analytics~=0.2.1',
	'azure-mgmt-datalake-store~=0.5.0',
	'azure-mgmt-datamigration~=10.0.0',
	'azure-mgmt-deploymentmanager~=0.2.0',
	'azure-mgmt-devtestlabs~=4.0',
	'azure-mgmt-dns~=8.0.0',
	'azure-mgmt-eventgrid==9.0.0',
	'azure-mgmt-eventhub~=9.1.0',
	'azure-mgmt-extendedlocation~=1.0.0b2',
	'azure-mgmt-hdinsight~=9.0.0',
	'azure-mgmt-imagebuilder~=1.0.0',
	'azure-mgmt-iotcentral~=9.0.0',
	'azure-mgmt-iothub==2.1.0',
	'azure-mgmt-iothubprovisioningservices~=1.0.0',
	'azure-mgmt-keyvault==9.3.0',
	'azure-mgmt-kusto~=0.3.0',
	'azure-mgmt-loganalytics~=12.0.0',
	'azure-mgmt-managedservices~=1.0',
	'azure-mgmt-managementgroups~=0.1',
	'azure-mgmt-maps~=2.0.0',
	'azure-mgmt-marketplaceordering==1.1.0',
	'azure-mgmt-media~=7.0',
	'azure-mgmt-monitor~=3.0.0',
	'azure-mgmt-msi~=0.2',
	'azure-mgmt-netapp~=5.1.0',
	'azure-mgmt-network~=19.3.0',
	'azure-mgmt-policyinsights~=1.0.0',
	'azure-mgmt-privatedns~=1.0.0',
	'azure-mgmt-rdbms~=10.0.0',
	'azure-mgmt-recoveryservicesbackup~=4.0.0',
	'azure-mgmt-recoveryservices~=2.0.0',
	'azure-mgmt-redhatopenshift==1.0.0',
	'azure-mgmt-redis~=13.1.0',
	'azure-mgmt-relay~=0.1.0',
	'azure-mgmt-reservations==0.6.0', # TODO: Use requirements.txt instead of '==' #9781
	'azure-mgmt-resource==20.0.0',
	'azure-mgmt-search~=8.0',
	'azure-mgmt-security~=2.0.0b1',
	'azure-mgmt-servicebus~=6.0.0',
	'azure-mgmt-servicefabricmanagedclusters~=1.0.0',
	'azure-mgmt-servicelinker==1.0.0b1',
	'azure-mgmt-servicefabric~=1.0.0',
	'azure-mgmt-signalr~=1.0.0b2',
	'azure-mgmt-sqlvirtualmachine~=1.0.0b1',
	'azure-mgmt-sql~=3.0.1',
	'azure-mgmt-storage~=19.0.0',
	'azure-mgmt-synapse~=2.1.0b2',
	'azure-mgmt-trafficmanager~=0.51.0',
	'azure-mgmt-web~=4.0.0',

I am afraid all of them may potentially break azure-cli.😥