bug: from_service_account_json fails without GOOGLE_APPLICATION_CREDENTIALS env var set

[ofek]

pypinfo-96fad88c6384.json

{
 "type": "service_account",
 "project_id": "pypinfo-167519",
 "private_key_id": "redacted",
 "private_key": "-----BEGIN PRIVATE KEY-----...redacted...-----END PRIVATE KEY-----\n",
 "client_email": "moa-379@pypinfo-167519.iam.gserviceaccount.com",
 "client_id": "109462256829445067003",
 "auth_uri": "https://accounts.google.com/o/oauth2/auth",
 "token_uri": "https://accounts.google.com/o/oauth2/token",
 "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
 "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/moa-379%40pypinfo-167519.iam.gserviceaccount.com"
}

Ofek@Ofek-PC ~ $ from google.cloud.bigquery import Client 
Ofek@Ofek-PC ~ $ client = Client.from_service_account_json(r'C:\Users\Ofek\AppData\Local\creds\pypinfo-96fad88c6384.json') 
xonsh: To log full traceback to a file set: $XONSH_TRACEBACK_LOGFILE = <filename> 
Traceback (most recent call last): 
 File "C:\Users\Ofek\Desktop\q\Miniconda3\lib\site-packages\xonsh\__amalgam__.py", line 14826, in default 
 run_compiled_code(code, self.ctx, None, 'single') 
 File "C:\Users\Ofek\Desktop\q\Miniconda3\lib\site-packages\xonsh\__amalgam__.py", line 2085, in run_compiled_code 
 func(code, glb, loc) 
 File "<xonsh-code>", line 1, in <module> 
 File "C:\Users\Ofek\Desktop\q\Miniconda3\lib\site-packages\google\cloud\client.py", line 71, in from_service_account_json 
 return cls(*args, **kwargs) 
 File "C:\Users\Ofek\Desktop\q\Miniconda3\lib\site-packages\google\cloud\bigquery\client.py", line 83, in __init__ 
 project=project, credentials=credentials, _http=_http) 
 File "C:\Users\Ofek\Desktop\q\Miniconda3\lib\site-packages\google\cloud\client.py", line 211, in __init__ 
 _ClientProjectMixin.__init__(self, project=project) 
 File "C:\Users\Ofek\Desktop\q\Miniconda3\lib\site-packages\google\cloud\client.py", line 167, in __init__ 
 project = self._determine_default(project) 
 File "C:\Users\Ofek\Desktop\q\Miniconda3\lib\site-packages\google\cloud\client.py", line 180, in _determine_default 
 return _determine_default_project(project) 
 File "C:\Users\Ofek\Desktop\q\Miniconda3\lib\site-packages\google\cloud\_helpers.py", line 181, in _determine_default_project 
 _, project = google.auth.default() 
 File "C:\Users\Ofek\Desktop\q\Miniconda3\lib\site-packages\google\auth\_default.py", line 282, in default 
 raise exceptions.DefaultCredentialsError(_HELP_MESSAGE) 
google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or 
explicitly create credential and re-run the application. For more 
information, please see 
https://developers.google.com/accounts/docs/application-default-credentials. 

[ofek]

edit: removed, as it wasn't the fix

[dhermes]

The error message is pretty specific. Maybe the issue is the way we're forming the query? @tswast is this an unexpected error message?

[tswast]

@ofek You shouldn't be setting project='the-psf' in the load from file function call. You should be setting the project for which you want the queries to be charged to (your project).

You can still query data in the project the-psf, so long as your project has access to those datasets.

[dhermes]

Thanks @tswast I figured it was something straightforward.

@ofek I'm pre-emptively closing this. Happy to re-open if there is still something unresolved.

[ofek]

@tswast Ok thanks, that makes sense and is what I originally tried in op.

@dhermes Can you please reopen this? See first post.

Do either of you know why from_service_account_json fails then?

[dhermes]

@ofek This is actually #1883 rearing it's ugly head. I've been meaning to tackle that one, this is just another bump that I should do it ASAP.

The issue is that the project is implicitly determined from the environment when it should be determined from the JSON file.

Sorry I conflated your two issues as one. Next time, it'd be better if you just filed two separate issues, though I realize that's friction you don't really want. We really appreciate you filing any bug at all!

[ofek]

@dhermes Oh, I see. Thanks! I suppose I'll just parse the file for project id myself to pass to from_service_account_json.

[dhermes]

@ofek Yeah unfortunately that's the best you can do for now. I'll try to send a legit fix for #1883 tomorrow.