Cannot use the azure-mgmt-resourcegraph python library on the CLI

[DevenJnando]

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug

Part 1 of the issue:

When using the version of azure-common which is bundled into azure-cli we hit the following error when attempting to obtain a list of Subscriptions through the get_client_from_cli_profile(SubscriptionClient) function:

File ".azdev/azdev-env/lib/python3.8/site-packages/azure/common/client_factory.py", line 100, in get_client_from_cli_profile return _instantiate_client(client_class, **parameters) File "azdev/azdev-env/lib/python3.8/site-packages/azure/common/client_factory.py", line 35, in _instantiate_client return client_class(**kwargs) TypeError: __init__() missing 1 required positional argument: 'credential'

This seems to be due to the fact that a credential argument is expected in the azure.mgmt.resource class, when a credentials argument is being provided in the get_client_from_cli_profile() function in azure.common

Part 2 of the issue:

If we update our dependencies manually to ensure that azure-common, azure-mgmt-resource and azure-mgmt-resourcegraph are all version 1.1.27, 19.0.0 and 8.0.0 respectively, we can consume the credentials change to credential. However, when attempting to iterate over a list of Subscriptions obtained using the get_client_from_cli_profile(SubscriptionClient) function, the following error is thrown:

File "/.azdev/azdev-env/lib/python3.8/site-packages/azure/common/credentials.py", line 72, in get_token _, token, fulltoken = credentials._token_retriever() # pylint:disable=protected-access TypeError: _retrieve_token() missing 1 required positional argument: 'token_resource'

To Reproduce

Reproduce part 1:
azure-mgmt-resource and azure-mgmt-resourcegraph should be listed as azdev dependencies and the following code should be run:

subsList = [] subsDict = [] subsClient = get_client_from_cli_profile(SubscriptionClient)

Reproduce part 2:

azure-common azure-mgmt-resource and azure-mgmt-resourcegraph should be listed as azdev dependencies using versions 1.1.27, 19.0.0 and 8.0.0 respectively, and the following lines of code should be run:

`subsList = []
subsDict = []
subsClient = get_client_from_cli_profile(SubscriptionClient)

for sub in subsClient.subscriptions.list():
subsDict.append(sub.as_dict())`

Expected behavior

Part 1:

It is expected that a subscription client should be returned.

Part 2:

It is expected that each subscription should be added to the subsDict list object.

Environment summary

OS - Linux Mint
OS version - 20.1 (Ulyssa)
CLI version - 2.28.0
CLI Core version - 2.28.0
Installation method - azdev
azure-mgmt-resource version - 19.0.0
azure-mgmt-resourcegraph version - 8.0.0

Part 1 only:
azure-common version - 1.1.22

Part 2 only:
azure-common version - 1.1.27

[yonzhan]

route to service team

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc Chirag Gupta [MSFT] (@chiragg4u).

Issue Details

---

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug

Part 1 of the issue:

When using the version of azure-common which is bundled into azure-cli we hit the following error when attempting to obtain a list of Subscriptions through the get_client_from_cli_profile(SubscriptionClient) function:

File ".azdev/azdev-env/lib/python3.8/site-packages/azure/common/client_factory.py", line 100, in get_client_from_cli_profile return _instantiate_client(client_class, **parameters) File "azdev/azdev-env/lib/python3.8/site-packages/azure/common/client_factory.py", line 35, in _instantiate_client return client_class(**kwargs) TypeError: __init__() missing 1 required positional argument: 'credential'

This seems to be due to the fact that a credential argument is expected in the azure.mgmt.resource class, when a credentials argument is being provided in the get_client_from_cli_profile() function in azure.common

Part 2 of the issue:

If we update our dependencies manually to ensure that azure-common, azure-mgmt-resource and azure-mgmt-resourcegraph are all version 1.1.27, 19.0.0 and 8.0.0 respectively, we can consume the credentials change to credential. However, when attempting to iterate over a list of Subscriptions obtained using the get_client_from_cli_profile(SubscriptionClient) function, the following error is thrown:

File "/.azdev/azdev-env/lib/python3.8/site-packages/azure/common/credentials.py", line 72, in get_token _, token, fulltoken = credentials._token_retriever() # pylint:disable=protected-access TypeError: _retrieve_token() missing 1 required positional argument: 'token_resource'

To Reproduce

Reproduce part 1:
azure-mgmt-resource and azure-mgmt-resourcegraph should be listed as azdev dependencies and the following code should be run:

subsList = [] subsDict = [] subsClient = get_client_from_cli_profile(SubscriptionClient)

Reproduce part 2:

azure-common azure-mgmt-resource and azure-mgmt-resourcegraph should be listed as azdev dependencies using versions 1.1.27, 19.0.0 and 8.0.0 respectively, and the following lines of code should be run:

`subsList = []
subsDict = []
subsClient = get_client_from_cli_profile(SubscriptionClient)

for sub in subsClient.subscriptions.list():
subsDict.append(sub.as_dict())`

Expected behavior

Part 1:

It is expected that a subscription client should be returned.

Part 2:

It is expected that each subscription should be added to the subsDict list object.

Environment summary

OS - Linux Mint
OS version - 20.1 (Ulyssa)
CLI version - 2.28.0
CLI Core version - 2.28.0
Installation method - azdev
azure-mgmt-resource version - 19.0.0
azure-mgmt-resourcegraph version - 8.0.0

Part 1 only:
azure-common version - 1.1.22

Part 2 only:
azure-common version - 1.1.27

Author:	DevenJnando
Assignees:	-
Labels:	Service Attention, Resource Graph
Milestone:	-

[jiasli]

Part 1: TypeError: init() missing 1 required positional argument: 'credential'

File ".azdev/azdev-env/lib/python3.8/site-packages/azure/common/client_factory.py", line 100, in get_client_from_cli_profile
  return _instantiate_client(client_class, **parameters) 
File "azdev/azdev-env/lib/python3.8/site-packages/azure/common/client_factory.py", line 35, in _instantiate_client 
  return client_class(**kwargs) 
TypeError: __init__() missing 1 required positional argument: 'credential'

azure-mgmt-resource v13.0.0 -> v15.0.0 is Track 1 -> Track 2 version bump and introduces several breaking changes. credentials -> credential is one of them. azure-mgmt-resource currently used by Azure CLI is now Track 2 (#17783) which uses credential.

Track 1 v13.0.0: https://github.com/Azure/azure-sdk-for-python/blob/azure-mgmt-resource_13.0.0/sdk/resources/azure-mgmt-resource/azure/mgmt/resource/subscriptions/_subscription_client.py#L57

class SubscriptionClient(SubscriptionClientOperationsMixin, MultiApiClientMixin, SDKClient):
    def __init__(self, credentials, api_version=None, base_url=None, profile=KnownProfiles.default):

Track 2 v15.0.0: https://github.com/Azure/azure-sdk-for-python/blob/azure-mgmt-resource_15.0.0/sdk/resources/azure-mgmt-resource/azure/mgmt/resource/subscriptions/_subscription_client.py#L57

class SubscriptionClient(SubscriptionClientOperationsMixin, MultiApiClientMixin, _SDKClient):
    def __init__(
        self,
        credential,  # type: "TokenCredential"

azure-common doesn't make the change to adapt to Track 2 SDKs accordingly.

Part 2: TypeError: _retrieve_token() missing 1 required positional argument: 'token_resource'

That change on _retrieve_token was made to implement Conditional Access with MFA (#17738).

Track 1 SDK doesn’t manage resource(scopes) - CLI manages resource, but Track 2 SDK manages scopes(resource) by itself. CLI made this change according to the Track 2 SDK design so that _token_retriever should always be called with a resource. This provides a consistent way to display a re-login message if MFA is required.

azure-common is incorrectly using the private method _token_retriever. Azure CLI doesn't guarantee the interface of private methods like _token_retriever.

https://github.com/Azure/azure-sdk-for-python/blob/7373db2a493e23bb03a4d8d2f74fff9767f0cb94/sdk/core/azure-common/azure/common/credentials.py#L72

_, token, fulltoken = credentials._token_retriever()  # pylint:disable=protected-access

The pylint error protected-access should serve as a sign of warning, but is ignored.

All incorrect usages have been removed from Azure CLI code: #17812

Instead, azure-common should call public methods like

1. azure.cli.core._profile.Profile.get_raw_token
2. azure.cli.core._profile.Profile.get_login_credentials
3. azure.identity.AzureCliCredential
4. subprocess az account get-access-token

In the MSAL migration (#18944), _token_retriever will be removed.

Solution

azure-common has been deprecated and out of maintenance.

The right way to create an SDK client is to call azure.cli.core.commands.client_factory.get_mgmt_service_client which automatically

• (part 1) handles Track 1 and Track 2 difference on credential
• (part 2) uses the correct public method azure.cli.core._profile.Profile.get_login_credentials to create a credential, which is used by SDK to get an access token