read/write Google Cloud Storage from local Google App Engine

[zdenulo]

short question: Is uploading/downloading to Google Cloud Storage from local Google App Engine working?

long question:
I get this error:

Traceback (most recent call last):
 File "/home/username/bin/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
 rv = self.handle_exception(request, response, e)
 File "/home/username/bin/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
 rv = self.router.dispatch(request, response)
 File "/home/username/bin/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
 return route.handler_adapter(request, response)
 File "/home/username/bin/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
 return handler.dispatch()
 File "/home/username/bin/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
 return self.handle_exception(e, self.app.debug)
 File "/home/username/bin/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
 return method(*args, **kwargs)
 File "/home/username/dev/myproject/main.py", line 89, in get
 b.upload_from_string('tesssssss')
 File "/home/username/.virtualenvs/fishlead/lib/python2.7/site-packages/google/cloud/storage/blob.py", line 967, in upload_from_string
 content_type=content_type, client=client)
 File "/home/username/.virtualenvs/fishlead/lib/python2.7/site-packages/google/cloud/storage/blob.py", line 891, in upload_from_file
 client, file_obj, content_type, size, num_retries)
 File "/home/username/.virtualenvs/fishlead/lib/python2.7/site-packages/google/cloud/storage/blob.py", line 818, in _do_upload
 client, stream, content_type, size, num_retries)
 File "/home/username/.virtualenvs/fishlead/lib/python2.7/site-packages/google/cloud/storage/blob.py", line 768, in _do_resumable_upload
 client, stream, content_type, size, num_retries)
 File "/home/username/.virtualenvs/fishlead/lib/python2.7/site-packages/google/cloud/storage/blob.py", line 727, in _initiate_resumable_upload
 total_bytes=size, stream_final=False)
 File "/home/username/.virtualenvs/fishlead/lib/python2.7/site-packages/google/resumable_media/requests/upload.py", line 326, in initiate
 retry_strategy=self._retry_strategy)
 File "/home/username/.virtualenvs/fishlead/lib/python2.7/site-packages/google/resumable_media/requests/_helpers.py", line 98, in http_request
 func, RequestsMixin._get_status_code, retry_strategy)
 File "/home/username/.virtualenvs/fishlead/lib/python2.7/site-packages/google/resumable_media/_helpers.py", line 146, in wait_and_retry
 response = func()
 File "/home/username/.virtualenvs/fishlead/lib/python2.7/site-packages/google/auth/transport/requests.py", line 179, in request
 method, url, data=data, headers=request_headers, **kwargs)
 File "/home/username/dev/myproject/lib/requests/sessions.py", line 502, in request
 resp = self.send(prep, **send_kwargs)
 File "/home/username/dev/myproject/lib/requests/sessions.py", line 612, in send
 r = adapter.send(request, **kwargs)
 File "/home/username/dev/myproject/lib/requests/adapters.py", line 490, in send
 raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', error(13, 'Permission denied'))

everything works ok when I deploy.
I tried different combinations of authentications based on https://google-cloud-python.readthedocs.io/en/latest/core/auth.html

Strange thing though is that I can for example list buckets in the project locally which means that I have (some) permissions/access. Service account which I am using has admin rights both for project and bucket.

I'm using Python 2.7.13
google-gcloud 0.26.1

here is repository with code which can be used for testing https://github.com/zdenulo/gcs_local_gae_error

[dhermes]

@jonparrott Is there some magical incantation needed for using requests on GAE (in this case the dev server)?

[theacodes]

Even with that we just fundamentally do not support App Engine standard. See #1893 for tracking standard support.

[zdenulo]

I realized just now, using requests-toolbelt (https://github.com/requests/toolbelt) solved the problem, i.e. adding:

from requests_toolbelt.adapters import appengine
appengine.monkeypatch()

in appengine_config.py file

Maybe it could be emphasized in documentation somewhere

[theacodes]

@zdenulo App Engine standard's docs call that out here: https://cloud.google.com/appengine/docs/standard/python/issue-requests

We don't document it here because we do not yet support standard.

(closing for now, but feel free to comment)

[zdenulo]

@jonparrott thanks for info