`az storage blob list` fails with "NoneType object is not callable" on Amazon Linux

[cartley]

Describe the bug

az storage blob list fails on Amazon Linux 2.

Command Name
az storage blob list

Errors:

The command failed with an unexpected error. Here is the traceback:
'NoneType' object is not callable
Traceback (most recent call last):
  File "/usr/lib64/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
    raise ex
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 712, in _run_job
    return cmd_copy.exception_handler(ex)
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/__init__.py", line 352, in new_handler
    raise ex
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
    result = cmd_copy(params)
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 112, in handler
    client = self.client_factory(self.cli_ctx, command_args) if self.client_factory else None
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/_client_factory.py", line 246, in cf_container_client
    return cf_blob_service(cli_ctx, kwargs).get_container_client(container=kwargs.pop('container_name', None))
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/_client_factory.py", line 229, in cf_blob_service
    return t_blob_service(account_url=account_url, credential=credential, **client_kwargs)
TypeError: 'NoneType' object is not callable

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• az storage blob list --account-name {} --account-key {} --container-name {}

Expected Behavior

List of blobs is returned without error

Environment Summary

Linux-4.14.231-173.361.amzn2.x86_64-x86_64-with-glibc2.2.5
Python 3.7.9
Installer: RPM

azure-cli 2.23.0

$ lsb_release -a
LSB Version:	:core-4.1-amd64:core-4.1-noarch
Distributor ID:	Amazon
Description:	Amazon Linux release 2 (Karoo)
Release:	2
Codename:	Karoo

$ az --version
azure-cli                         2.23.0

core                              2.23.0
telemetry                          1.0.6

Python location '/usr/bin/python3'
Extensions directory '/home/hbl2731/.azure/cliextensions'

Python (Linux) 3.7.9 (default, Feb 18 2021, 03:10:35) 
[GCC 7.3.1 20180712 (Red Hat 7.3.1-12)]

Additional Context

I installed with RPM after following the instructions to use yum/dnf did not work. Yum failed wieth "Processing Dependency: libpython3.6m.so.1.0" even though system has Python 3.7.

yumdownloader azure-cli
rpm -ivh --nodeps azure-cli-2.23.0-1.el7.x86_64.rpm 

[yonzhan]

storage

[Juliehzl]

Hi Craig Artley (@cartley), could you show me your python package information with pip list? I am afraid that you are not using specified python package version in your system.
If your azure-multiapi-storage version is out of date, please upgrade to 0.6.0 version.

If you still get the error, could you add --debug in your command and show me debug info?

[cartley]

Sure, as far as I know I'm using the standard python3 for this system. Do I need to run this in a virtual environment?

~ $ pip list
bash: pip: command not found
~ $ pip3 list
Package    Version
---------- -------
pip        9.0.3  
setuptools 38.4.0 
~ $ pip3 list --user
~ $

~ $ yum list python3 python
Loaded plugins: amzn_workspaces_filter_updates, halt_os_update_check, priorities, update-motd
Installed Packages
python.x86_64                               2.7.18-1.amzn2.0.3                               @amzn2-core
python3.x86_64                              3.7.9-1.amzn2.0.2                                @amzn2-core
Available Packages
python3.i686                                3.7.9-1.amzn2.0.2                                amzn2-core 

[cartley]

Here is --debug output.

~ $ az --debug storage blob list --account-name XXXX --account-key XXXX --container-name XXXX
cli.knack.cli: Command arguments: ['--debug', 'storage', 'blob', 'list', '--account-name', 'XXXX', '--account-key', 'XXXX', '--container-name', 'XXXX']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f4814418dd0>, <function OutputProducer.on_global_arguments at 0x7f4813e6cf80>, <function CLIQuery.on_global_arguments at 0x7f4813e8def0>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: No module found from index for '['--debug', 'storage', 'blob', 'list', '--account-name', 'XXXX', '--account-key', 'XXXX', '--container-name', 'XXXX']'
cli.azure.cli.core: Loading all modules and extensions
cli.azure.cli.core: Discovered command modules: ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'config', 'configure', 'consumption', 'container', 'cosmosdb', 'databoxedge', 'deploymentmanager', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'interactive', 'iot', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'marketplaceordering', 'monitor', 'natgateway', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'reservations', 'resource', 'role', 'search', 'security', 'servicebus', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'synapse', 'util', 'vm']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: acr                       0.017        27       113
cli.azure.cli.core: acs                       0.423         7        51
cli.azure.cli.core: advisor                   0.002         3         6
cli.azure.cli.core: ams                       0.009        20        90
cli.azure.cli.core: apim                      0.006         9        50
cli.azure.cli.core: appconfig                 0.005         7        33
cli.azure.cli.core: appservice                0.021        62       221
cli.azure.cli.core: aro                       0.009         1         7
cli.azure.cli.core: backup                    0.007        13        49
cli.azure.cli.core: batch                     0.139        31        90
cli.azure.cli.core: batchai                   0.005        10        32
cli.azure.cli.core: billing                   0.011        19        52
cli.azure.cli.core: botservice                0.005        12        42
cli.azure.cli.core: cdn                       0.012        39       132
cli.azure.cli.core: cloud                     0.003         1         7
cli.azure.cli.core: cognitiveservices         0.003         5        17
cli.azure.cli.core: config                    0.002         2         7
cli.azure.cli.core: configure                 0.002         3         9
cli.azure.cli.core: consumption               0.004         8         9
cli.azure.cli.core: container                 0.003         1        11
cli.azure.cli.core: cosmosdb                  0.016        33       126
cli.azure.cli.core: databoxedge               0.006         5        27
cli.azure.cli.core: deploymentmanager         0.004         7        30
cli.azure.cli.core: dla                       0.007        23        62
cli.azure.cli.core: dls                       0.005         7        41
cli.azure.cli.core: dms                       0.004         3        22
cli.azure.cli.core: eventgrid                 0.007        18        61
cli.azure.cli.core: eventhubs                 0.006        14        51
cli.azure.cli.core: extension                 0.002         1         7
cli.azure.cli.core: feedback                  0.001         1         1
cli.azure.cli.core: find                      0.001         1         1
cli.azure.cli.core: hdinsight                 0.005         7        36
cli.azure.cli.core: interactive               0.001         1         1
cli.azure.cli.core: iot                       0.021        15        68
cli.azure.cli.core: keyvault                  0.030        19       117
cli.azure.cli.core: kusto                     0.003         3        14
cli.azure.cli.core: lab                       0.008        11        34
cli.azure.cli.core: managedservices           0.002         3         8
cli.azure.cli.core: maps                      0.002         3         7
cli.azure.cli.core: marketplaceordering       0.003         1         2
cli.azure.cli.core: monitor                   0.019        32       133
cli.azure.cli.core: natgateway                0.002         3         6
cli.azure.cli.core: netappfiles               0.057        13        54
cli.azure.cli.core: network                   0.169       133       615
cli.azure.cli.core: policyinsights            0.003         6        12
cli.azure.cli.core: privatedns                0.007        14        66
cli.azure.cli.core: profile                   0.003         2         9
cli.azure.cli.core: rdbms                     0.120        43       188
cli.azure.cli.core: redis                     0.003         4        24
cli.azure.cli.core: relay                     0.005        10        37
cli.azure.cli.core: reservations              0.003         5        12
cli.azure.cli.core: resource                  0.020        37       176
cli.azure.cli.core: role                      0.008        17        61
cli.azure.cli.core: search                    0.003         7        22
cli.azure.cli.core: security                  0.008        36        81
cli.azure.cli.core: servicebus                0.008        17        64
cli.azure.cli.core: servicefabric             0.008        27        77
cli.azure.cli.core: signalr                   0.003         5        16
cli.azure.cli.core: sql                       0.045        42       169
cli.azure.cli.core: sqlvm                     0.005         4        17
cli.azure.cli.core: storage                   0.072        52       242
cli.azure.cli.core: synapse                   0.015        34       143
cli.azure.cli.core: util                      0.002         2         4
cli.azure.cli.core: vm                        0.058        44       225
cli.azure.cli.core: Total (64)                1.469      1045      4197
cli.azure.cli.core: Loaded 1035 groups, 4197 commands.
cli.azure.cli.core: Updated command index in 0.006 seconds.
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f48100d2560>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/home/XXXX/.azure/commands/2021-05-21.10-51-26.unknown_command.11961.log'.
az_command_data_logger: command args: --debug {} {} {} --account-name {} --account-key {} --container-name {}
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7f480f0b8830>, <function register_global_query_examples_argument.<locals>.register_query_examples at 0x7f480ef7cd40>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7f480ef7cdd0>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7f480ef7cef0>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x7f480d248c20>]
The command requires the extension storage-blob-preview. Do you want to install it now? The command will continue to run after the extension is installed. (Y/n): y
cli.azure.cli.core.extension.dynamic_install: Run 'az config set extension.use_dynamic_install=yes_without_prompt' to allow installing extensions without prompt.
urllib3.connectionpool: Starting new HTTPS connection (1): aka.ms:443
urllib3.connectionpool: https://aka.ms:443 "GET /azure-cli-extension-index-v1 HTTP/1.1" 301 0
urllib3.connectionpool: Starting new HTTPS connection (1): azcliextensionsync.blob.core.windows.net:443
urllib3.connectionpool: https://azcliextensionsync.blob.core.windows.net:443 "GET /index1/index.json HTTP/1.1" 200 932197
cli.azure.cli.core.extension._resolve: Candidates ['storage_blob_preview-0.1.0-py2.py3-none-any.whl', 'storage_blob_preview-0.2.0-py2.py3-none-any.whl', 'storage_blob_preview-0.3.0-py2.py3-none-any.whl', 'storage_blob_preview-0.4.0-py2.py3-none-any.whl', 'storage_blob_preview-0.4.1-py2.py3-none-any.whl', 'storage_blob_preview-0.5.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['storage_blob_preview-0.1.0-py2.py3-none-any.whl', 'storage_blob_preview-0.2.0-py2.py3-none-any.whl', 'storage_blob_preview-0.3.0-py2.py3-none-any.whl', 'storage_blob_preview-0.4.0-py2.py3-none-any.whl', 'storage_blob_preview-0.4.1-py2.py3-none-any.whl', 'storage_blob_preview-0.5.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['storage_blob_preview-0.1.0-py2.py3-none-any.whl', 'storage_blob_preview-0.2.0-py2.py3-none-any.whl', 'storage_blob_preview-0.3.0-py2.py3-none-any.whl', 'storage_blob_preview-0.4.0-py2.py3-none-any.whl', 'storage_blob_preview-0.4.1-py2.py3-none-any.whl', 'storage_blob_preview-0.5.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Candidates ['storage_blob_preview-0.5.0-py2.py3-none-any.whl', 'storage_blob_preview-0.4.1-py2.py3-none-any.whl', 'storage_blob_preview-0.4.0-py2.py3-none-any.whl', 'storage_blob_preview-0.3.0-py2.py3-none-any.whl', 'storage_blob_preview-0.2.0-py2.py3-none-any.whl', 'storage_blob_preview-0.1.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension._resolve: Choosing the latest of the remaining candidates.
cli.azure.cli.core.extension._resolve: Chosen {'downloadUrl': 'https://azcliprod.blob.core.windows.net/cli-extensions/storage_blob_preview-0.5.0-py2.py3-none-any.whl', 'filename': 'storage_blob_preview-0.5.0-py2.py3-none-any.whl', 'metadata': {'azext.isPreview': True, 'azext.minCliCoreVersion': '2.16.0', 'classifiers': ['Development Status :: 4 - Beta', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'Programming Language :: Python', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.8', 'License :: OSI Approved :: MIT License'], 'extensions': {'python.details': {'contacts': [{'email': 'azpycli@microsoft.com', 'name': 'Microsoft Corporation', 'role': 'author'}], 'document_names': {'description': 'DESCRIPTION.rst'}, 'project_urls': {'Home': 'https://github.com/Azure/azure-cli-extensions'}}}, 'generator': 'bdist_wheel (0.30.0)', 'license': 'MIT', 'metadata_version': '2.0', 'name': 'storage-blob-preview', 'summary': 'Microsoft Azure Command-Line Tools Storage-blob-preview Extension', 'version': '0.5.0'}, 'sha256Digest': 'a3303da3925f8ad83c8eeced02f6b8897b355a3cc249424a7e70c621ee0d3642'}
cli.azure.cli.core.extension.operations: Extension source is url? True
cli.azure.cli.core.extension.operations: Downloading https://azcliprod.blob.core.windows.net/cli-extensions/storage_blob_preview-0.5.0-py2.py3-none-any.whl to /tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl
urllib3.connectionpool: Starting new HTTPS connection (1): azcliprod.blob.core.windows.net:443
urllib3.connectionpool: https://azcliprod.blob.core.windows.net:443 "GET /cli-extensions/storage_blob_preview-0.5.0-py2.py3-none-any.whl HTTP/1.1" 200 920814
cli.azure.cli.core.extension.operations: Downloaded to /tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl
cli.azure.cli.core.extension.operations: Validating the extension /tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl
cli.azure.cli.core.extension.operations: Checksum of /tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl is OK
cli.azure.cli.core.extension.operations: Validation successful on /tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl
cli.azure.cli.core.extension.operations: Linux distro check: Reading from: /etc/apt/sources.list.d/azure-cli.list
cli.azure.cli.core.extension.operations: Linux distro check: An error occurred while checking linux distribution version source list consistency.
cli.azure.cli.core.extension.operations: [Errno 2] No such file or directory: '/etc/apt/sources.list.d/azure-cli.list'
cli.azure.cli.core.extension.operations: Executing pip with args: ['install', '--target', '/home/XXXX/.azure/cliextensions/storage-blob-preview', '/tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl']
cli.azure.cli.core.extension.operations: Running: ['/usr/bin/python3', '-m', 'pip', 'install', '--target', '/home/XXXX/.azure/cliextensions/storage-blob-preview', '/tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']
cli.azure.cli.core.extension.operations: Using pip 21.0.1 from /usr/lib64/az/lib/python3.6/site-packages/pip (python 3.7)
Non-user install due to --prefix or --target option
Created temporary directory: /tmp/pip-target-1is8ac7z
Created temporary directory: /tmp/pip-ephem-wheel-cache-_crqvmrz
Created temporary directory: /tmp/pip-req-tracker-jtnb1fgr
Initialized build tracking at /tmp/pip-req-tracker-jtnb1fgr
Created build tracker: /tmp/pip-req-tracker-jtnb1fgr
Entered build tracker: /tmp/pip-req-tracker-jtnb1fgr
Created temporary directory: /tmp/pip-install-50e4lhzr
Processing /tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl
  Added storage-blob-preview==0.5.0 from file:///tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl to build tracker '/tmp/pip-req-tracker-jtnb1fgr'
  Removed storage-blob-preview==0.5.0 from file:///tmp/tmpjltjx141/storage_blob_preview-0.5.0-py2.py3-none-any.whl from build tracker '/tmp/pip-req-tracker-jtnb1fgr'
Created temporary directory: /tmp/pip-unpack-js_zo3od
Installing collected packages: storage-blob-preview

Successfully installed storage-blob-preview-0.5.0
Removed build tracker: '/tmp/pip-req-tracker-jtnb1fgr'

cli.azure.cli.core.extension.operations: Saved the whl to /home/XXXX/.azure/cliextensions/storage-blob-preview/storage_blob_preview-0.5.0-py2.py3-none-any.whl
cli.azure.cli.core.extension.operations: The installed extension 'storage-blob-preview' is in preview.
cli.azure.cli.core: Command index has been invalidated.
The command failed with an unexpected error. Here is the traceback:
'NoneType' object is not iterable
Traceback (most recent call last):
  File "/usr/lib64/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 556, in execute
    self.commands_loader.load_arguments(command)
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 517, in load_arguments
    loader.load_arguments(command)  # this adds entries to the argument registries
  File "/home/XXXX/.azure/cliextensions/storage-blob-preview/azext_storage_blob_preview/__init__.py", line 33, in load_arguments
    load_arguments(self, command)
  File "/home/XXXX/.azure/cliextensions/storage-blob-preview/azext_storage_blob_preview/_params.py", line 302, in load_arguments
    help=sas_help.format(get_permission_help_string(t_blob_permissions)),
  File "/home/XXXX/.azure/cliextensions/storage-blob-preview/azext_storage_blob_preview/_validators.py", line 624, in get_permission_help_string
    return ' '.join(['({}){}'.format(x[0], x[1:]) for x in allowed_values])
TypeError: 'NoneType' object is not iterable
To open an issue, please run: 'az feedback'
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f48100e20e0>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 30.187 seconds (init: 0.237, invoke: 29.950)
telemetry.save: Save telemetry record of length 3215 in cache
telemetry.check: Negative: The /home/XXXX/.azure/telemetry.txt was modified at 2021-05-21 10:51:15.648255, which in less than 600.000000 s