Skip to content

Support blob streaming for file-like objects - #1914

Merged
tseaver merged 3 commits into
googleapis:masterfrom
linar-jether:master
Jul 14, 2016
Merged

Support blob streaming for file-like objects#1914
tseaver merged 3 commits into
googleapis:masterfrom
linar-jether:master

Conversation

@linar-jether

@linar-jetherlinar-jether commented Jun 27, 2016

Copy link
Copy Markdown
Contributor

When uploading file-like objects without a specified size, upload in chunks instead of exhausting the stream and keeping the data in-memory.

See #1909

@googlebot

googlebot commented Jun 27, 2016

Copy link
Copy Markdown

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
  • If you signed the CLA as a corporation, please let us know the company's name.

@googlebotgooglebot added the cla: no This human has *not* signed the Contributor License Agreement. label Jun 27, 2016
@linar-jether

linar-jether commented Jun 27, 2016

Copy link
Copy Markdown
ContributorAuthor

Signed it

@googlebot

googlebot commented Jun 27, 2016

Copy link
Copy Markdown

CLAs look good, thanks!

@googlebotgooglebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Jun 27, 2016
@tseavertseaver added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. api: storage Issues related to the Cloud Storage API. labels Jun 27, 2016
tseaver
tseaver reviewed Jul 11, 2016
View reviewed changes
Comment threadgcloud/streaming/buffered_stream.py Outdated
self._buffered_data = self._stream.read(size)
self._buffered_data = self._stream.read(size) \
if not hasattr(self._stream,
'closed') or not self._stream.closed else ''

This comment was marked as spam.

Sign in to view

@linar-jether

linar-jether commented Jul 12, 2016

Copy link
Copy Markdown
ContributorAuthor

fixed according to comments

@tseaver

tseaver commented Jul 12, 2016

Copy link
Copy Markdown
Contributor

@linar-jether thanks for the changes. The pylint failures on Travis aren't due to this PR: they released a new version last Friday, which introduced new checks that we now fail. The master branch now has fixes for those failures: could you rebase your branch against it and re-push?

@linar-jether

linar-jether commented Jul 13, 2016

Copy link
Copy Markdown
ContributorAuthor

All tests passed

daspecster
daspecster reviewed Jul 13, 2016
View reviewed changes
Comment threadgcloud/storage/blob.py Outdated
import copy
import hashlib
from io import BytesIO
from io import BytesIO, UnsupportedOperation

This comment was marked as spam.

Sign in to view

@daspecster

daspecster commented Jul 13, 2016

Copy link
Copy Markdown
Contributor

@linar-jether, cool! Do you think you could squash these commits down?
Also, I noticed the imports issue.

Otherwise LGTM, but I defer to @tseaver/@dhermes for confirmation.

@linar-jether
linar-jetherforce-pushed the master branch 2 times, most recently from 742a95e to 43464b7CompareJuly 14, 2016 08:29
@linar-jether

linar-jether commented Jul 14, 2016

Copy link
Copy Markdown
ContributorAuthor

Sure, squashed all previous commits and separated the imports

@tseaver
tseaver merged commit bd6435a into googleapis:masterJul 14, 2016
@tseavertseaver mentioned this pull request Jul 14, 2016
Closed
@tseaver

tseaver commented Jul 14, 2016

Copy link
Copy Markdown
Contributor

@linar-jether Thanks again for your work!

This was referenced Jul 18, 2016
Merged
Merged
This was referenced Aug 3, 2016
Closed
Merged
@linar-jetherlinar-jether mentioned this pull request Jun 27, 2017
Closed
parthea pushed a commit that referenced this pull request Mar 6, 2026
@Linchin@tswast
fix: add pyarrow version check for range support (#1914)
41419d7
* fix: add pyarrow version check for range support
* add comment why we are making a separate constant
---------
Co-authored-by: Tim Sweña (Swast) <swast@google.com>
parthea pushed a commit that referenced this pull request Apr 1, 2026
@jialuoo
fix: resolve location reset issue in bigquery options (#1914)
dd7d61e
* fix: resolve location reset issue in bigquery options
* fix
* fix
* fix the test
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

api: storageIssues related to the Cloud Storage API.cla: yesThis human has signed the Contributor License Agreement.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@linar-jether@googlebot@tseaver@daspecster