`ModuleNotFoundError` warnings in `az self-test --debug`

[bebound]

Describe the bug

Lots of ModuleNotFoundError: No module named 'azure.mgmt.keyvault.v2023_02_01.azure' errors are raised when running az self-test. This occurs as early as 2.30.0

cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
  File "c:\users\kk\developer\azure-cli\src\azure-cli-core\azure\cli\core\profiles\_shared.py", line 651, in _get_attr
    op = import_module(full_mod_path)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\kk\AppData\Local\Programs\Python\Python311\Lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1142, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'azure.mgmt.keyvault.v2023_02_01.azure'

In

azure-cli/src/azure-cli-core/azure/cli/core/__init__.py

Line 709 in 68990c9

model = self.get_models(doc_string_source)

,
if doc_string_source is azure.mgmt.keyvault.v2023_02_01.models#VaultProperties, when it calls

azure-cli/src/azure-cli-core/azure/cli/core/profiles/_shared.py

Lines 646 to 651 in 68990c9

	def _get_attr(sdk_path, mod_attr_path, checked=True):
	try:
	attr_mod, attr_path = mod_attr_path.split('#') \
	if '#' in mod_attr_path else (mod_attr_path, '')
	full_mod_path = '{}.{}'.format(sdk_path, attr_mod) if attr_mod else sdk_path
	op = import_module(full_mod_path)

,
the full_mod_path become azure.mgmt.keyvault.v2023_02_01.azure.mgmt.keyvault.v2023_02_01.models, and it fails to import.

[yonzhan]

Thank you for opening this issue, we will look into it.

[azure-client-tools-bot-prd]

Hi Hang (@bebound)
Find similar issue #19640.

Issue title	Unofficial Azure CLI deb packages are distributed by Linux repositories
Create time	2021-09-22
Comment number	9

Possible solution:
The error message ModuleNotFoundError: No module named 'azure.mgmt.keyvault.v2023_02_01.azure' is raised when running az self-test. This occurs as early as version 2.30.0. The issue is caused by the full_mod_path becoming azure.mgmt.keyvault.v2023_02_01.azure.mgmt.keyvault.v2023_02_01.models, which fails to import.

To resolve this issue, you can try installing the latest official Azure CLI from the official source, which is https://packages.microsoft.com/. Please follow the official document to install the latest official Azure CLI: https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt.

If you have already installed the unofficial Azure CLI, you may uninstall it first with sudo apt remove azure-cli --yes && sudo apt autoremove --yes. Then install from the official source.

---

Please confirm if this resolves your issue.

[jiasli]

#19030 made azure.cli.core.profiles._shared._get_attr print the call stack in order to provide information on why ModuleNotFoundError(ImportError) is raised.

For example, when importing Azure Storage data-plane SDK azure.multiapi.storage, either scenario can raise ImportError:

1. Underlying dependency _cffi_backend library is missing
2. A model is missing from a certain API version used by Azure Stack

Without logging the call stack, we are actually discarding information by replacing it with a None. This can make debugging very difficult in some cases (#18159).

This actually brings up a deeper discussion - whether we want to make --debug log that verbose, so that it even includes program execution traces. Unfortunately, Python's logging system doesn't support TRACE logging level: https://docs.python.org/3/library/logging.html#logging-levels. We are facing a dilemma here:

• Providing too much information in debug log makes the log flooded with useless information.
• Providing too little information can cause information loss.

Finding the sweet spot can be challenging in real life.

If you feel the debug log printed by _get_attr is distracting, you may either

• Revert {Core} Log _get_attr exceptions #19030 by commenting those lines. Only turn them on during troubleshooting.
• Only print the error message, but this also means we will lose information.