bad client behavior when loading a service account key or token

[rvandegrift]

The storage client doesn't behave correctly when a service account key or token is supplied via the credentials parameter, but the project_id is not provided. In this case, it falls backs to searching for credentials.

There's two issues with the implementation:

1. The project_id docstring says it will be inferred from the environment if omitted. If credentials contains a service account key, that search should include the data in the key (it contains the correct project id).

2. If other credentials are found, the client silently loads them. This is very surprising, and could result in security issues when multiple credentials are available. (I discovered this when the client autoloaded my SDK credentials despite being provided a service account key.)

Ideally, if credentials is not None, the client should disable all searches and just use the provided info. If those creds fail, subsequent requests should fail.

If this is impossible due to being an API incompatibility, then a warning should be issued to tell the user that the provided credentials were ignored.

Environment details

• OS type and version: Debian buster
• Python version: Python 3.7.7
• pip version: pip 20.1.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
• google-cloud-storage version: 1.29.0

Steps to reproduce

1. Create a service account and a key or token.
2. Instantiate a client using that credential without specifying project_id.

Code example

Example showing that it falls back to default search even when a key is provided:

>>> from google.cloud import storage
>>> from google.oauth2 import service_account
>>> credentials = service_account.Credentials.from_service_account_file('gcs-test.json')
>>> storage_client = storage.Client(credentials=credentials)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/usr/local/lib/python3.7/site-packages/google/cloud/storage/client.py", line 111, in __init__
 project=project, credentials=credentials, _http=_http
 File "/usr/local/lib/python3.7/site-packages/google/cloud/client.py", line 226, in __init__
 _ClientProjectMixin.__init__(self, project=project)
 File "/usr/local/lib/python3.7/site-packages/google/cloud/client.py", line 178, in __init__
 project = self._determine_default(project)
 File "/usr/local/lib/python3.7/site-packages/google/cloud/client.py", line 193, in _determine_default
 return _determine_default_project(project)
 File "/usr/local/lib/python3.7/site-packages/google/cloud/_helpers.py", line 186, in _determine_default_project
 _, project = google.auth.default()
 File "/usr/local/lib/python3.7/site-packages/google/auth/_default.py", line 321, in default
 raise exceptions.DefaultCredentialsError(_HELP_MESSAGE)
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started

To see the silent fallback, run this in an environment where you've done gcloud auth application-default.

[HemangChothani]

@rvandegrift The related feature request has already been opened which you can find https://github.com/googleapis/python-cloud-core/issues/8.
I'm closing this issue because it's not only a Storage library request as it would update a common library shared other product libraries as well.