Swapping out google.streaming for google-resumable-media in GCS downloads. by dhermes · Pull Request #3323 · googleapis/google-cloud-python

dhermes · 2017-04-24T18:32:15Z

Check out https://googlecloudplatform.github.io/google-resumable-media-python/0.0.2/ for details on the new library.

…oads.

Sign in to view


 REQUIREMENTS = [
 'google-cloud-core >= 0.24.0, < 0.25dev',
+ 'google-auth',


Sign in to view

 'google-cloud-core >= 0.24.0, < 0.25dev',
+ 'google-auth',
+ 'google-resumable-media == 0.0.2',
+ 'requests',


Sign in to view

 import six
 from six.moves.urllib.parse import quote

+import google.auth.transport.requests as tr_requests


Sign in to view

 import six
 from six.moves.urllib.parse import quote

+import google.auth.transport.requests as tr_requests


Sign in to view

+ if self.chunk_size is None:
+ download = resumable_media.Download(download_url, headers=headers)
+ response = download.consume(transport)
+ file_obj.write(response.content)


Sign in to view

- download.chunksize = self.chunk_size
-
+ # Get the download URL.
+ download_url = self._get_download_url()


dhermes · 2017-04-24T22:35:48Z

I filed googleapis/google-resumable-media-python#5 to carry on the discussion about streaming the contents in a more memory-efficient manner (when possible).

For now, I am going to merge this based on an LGTM from @jonparrott (given in chat)

…oads. (googleapis#3323) * Swapping out google.streaming for google-resumable-media in GCS downloads. * Dropping tr_requests import alias in storage blob module. * Putting lower bounds on new deps in storage package.

…oads. (#3323) * Swapping out google.streaming for google-resumable-media in GCS downloads. * Dropping tr_requests import alias in storage blob module. * Putting lower bounds on new deps in storage package.

Swapping out google.streaming for google-resumable-media in GCS downl…
9756c37
…oads.

dhermes added the api: storage Issues related to the Cloud Storage API. label Apr 24, 2017

dhermes assigned lukesneeringer and tseaver Apr 24, 2017

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 24, 2017

theacodes reviewed Apr 24, 2017
View reviewed changes

tseaver reviewed Apr 24, 2017
View reviewed changes

dhermes added 2 commits April 24, 2017 14:44

Dropping tr_requests import alias in storage blob module.
4b178b2

Putting lower bounds on new deps in storage package.
970506b

dhermes mentioned this pull request Apr 24, 2017
Should stream / iter_contents be used when using Downloadgoogleapis/google-resumable-media-python#5
Closed

dhermes merged commit d0ef6a9 into googleapis:masterApr 24, 2017

dhermes deleted the resumable-media-for-GCS-downloads branch April 24, 2017 22:36

This was referenced Apr 27, 2017
Cut releases of core, error reporting, pubsub, spanner and storage. #3340
Merged
Error when downloading blobs with storage v1.1.0 #3342
Closed

Uh oh!

Conversation

dhermes commented Apr 24, 2017

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

dhermes commented Apr 24, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants