Skip to content
This repository was archived by the owner on Mar 6, 2026. It is now read-only.

perf: add size parameter for load table from dataframe and json methods - #280

Merged
HemangChothani merged 8 commits into
googleapis:masterfrom
MaxxleLLC:bigquery_issue_277
Oct 14, 2020
Merged

perf: add size parameter for load table from dataframe and json methods#280
HemangChothani merged 8 commits into
googleapis:masterfrom
MaxxleLLC:bigquery_issue_277

Conversation

@HemangChothani

Copy link
Copy Markdown
Contributor

Fixes#277

Also applied on load_table_from_json method.

@HemangChothani
HemangChothani requested review from a team, plamut and tswastSeptember 29, 2020 05:23
@google-clagoogle-claBot added the cla: yes This human has signed the Contributor License Agreement. label Sep 29, 2020
@product-auto-labelproduct-auto-labelBot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Sep 29, 2020

@tswasttswast left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: of the valid conventional commits at https://github.com/googleapis/release-please/blob/master/src/releasers/python.ts, perf: is more appropriate in this case, as we aren't adding any new features.

Comment threadgoogle/cloud/bigquery/client.py Outdated
return self.load_table_from_file(
data_file,
destination,
size=len(data_str),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This length needs to be on the encoded string, not the unicode string, otherwise the size will be wrong when non-ASCII characters are included.

@HemangChothaniHemangChothani changed the title feat: add size parameter for load from dataframe and jsonpref: add size parameter for load table from dataframe and json methodsSep 30, 2020
@tswasttswast changed the title pref: add size parameter for load table from dataframe and json methodsperf: add size parameter for load table from dataframe and json methodsSep 30, 2020

@tswasttswast left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Let's wait for #278 (2.0 release) before merging this PR.

@HemangChothani

Copy link
Copy Markdown
ContributorAuthor

Unit test failed is not related to the changes, issue will be resolved by PR #295

@tseaver

Copy link
Copy Markdown
Contributor

@tswast Any reason not to merge this now?

@tswast

Copy link
Copy Markdown
Contributor

Any reason not to merge this now?

It actually won't let me merge. Maybe "allow edits from maintainers" is not enabled?

Screen Shot 2020-10-13 at 4 07 22 PM

@HemangChothani
HemangChothani merged commit 3be78b7 into googleapis:masterOct 14, 2020
gcf-merge-on-greenBot pushed a commit that referenced this pull request Oct 19, 2020
🤖 I have created a release \*beep\* \*boop\* ---
## [2.2.0](https://www.github.com/googleapis/python-bigquery/compare/v2.1.0...v2.2.0) (2020-10-19)
### Features
* add method api_repr for table list item ([#299](https://www.github.com/googleapis/python-bigquery/issues/299)) ([07c70f0](https://www.github.com/googleapis/python-bigquery/commit/07c70f0292f9212f0c968cd5c9206e8b0409c0da))
* add support for listing arima, automl, boosted tree, DNN, and matrix factorization models ([#328](https://www.github.com/googleapis/python-bigquery/issues/328)) ([502a092](https://www.github.com/googleapis/python-bigquery/commit/502a0926018abf058cb84bd18043c25eba15a2cc))
* add timeout paramter to load_table_from_file and it dependent methods ([#327](https://www.github.com/googleapis/python-bigquery/issues/327)) ([b0dd892](https://www.github.com/googleapis/python-bigquery/commit/b0dd892176e31ac25fddd15554b5bfa054299d4d))
* add to_api_repr method to Model ([#326](https://www.github.com/googleapis/python-bigquery/issues/326)) ([fb401bd](https://www.github.com/googleapis/python-bigquery/commit/fb401bd94477323bba68cf252dd88166495daf54))
* allow client options to be set in magics context ([#322](https://www.github.com/googleapis/python-bigquery/issues/322)) ([5178b55](https://www.github.com/googleapis/python-bigquery/commit/5178b55682f5e264bfc082cde26acb1fdc953a18))
### Bug Fixes
* make TimePartitioning repr evaluable ([#110](https://www.github.com/googleapis/python-bigquery/issues/110)) ([20f473b](https://www.github.com/googleapis/python-bigquery/commit/20f473bfff5ae98377f5d9cdf18bfe5554d86ff4)), closes [#109](https://www.github.com/googleapis/python-bigquery/issues/109)
* use version.py instead of pkg_resources.get_distribution ([#307](https://www.github.com/googleapis/python-bigquery/issues/307)) ([b8f502b](https://www.github.com/googleapis/python-bigquery/commit/b8f502b14f21d1815697e4d57cf1225dfb4a7c5e))
### Performance Improvements
* add size parameter for load table from dataframe and json methods ([#280](https://www.github.com/googleapis/python-bigquery/issues/280)) ([3be78b7](https://www.github.com/googleapis/python-bigquery/commit/3be78b737add7111e24e912cd02fc6df75a07de6))
### Documentation
* update clustering field docstrings ([#286](https://www.github.com/googleapis/python-bigquery/issues/286)) ([5ea1ece](https://www.github.com/googleapis/python-bigquery/commit/5ea1ece2d911cdd1f3d9549ee01559ce8ed8269a)), closes [#285](https://www.github.com/googleapis/python-bigquery/issues/285)
* update snippets samples to support version 2.0 ([#309](https://www.github.com/googleapis/python-bigquery/issues/309)) ([61634be](https://www.github.com/googleapis/python-bigquery/commit/61634be9bf9e3df7589fc1bfdbda87288859bb13))
### Dependencies
* add protobuf dependency ([#306](https://www.github.com/googleapis/python-bigquery/issues/306)) ([cebb5e0](https://www.github.com/googleapis/python-bigquery/commit/cebb5e0e911e8c9059bc8c9e7fce4440e518bff3)), closes [#305](https://www.github.com/googleapis/python-bigquery/issues/305)
* require pyarrow for pandas support ([#314](https://www.github.com/googleapis/python-bigquery/issues/314)) ([801e4c0](https://www.github.com/googleapis/python-bigquery/commit/801e4c0574b7e421aa3a28cafec6fd6bcce940dd)), closes [#265](https://www.github.com/googleapis/python-bigquery/issues/265)
---
This PR was generated with [Release Please](https://github.com/googleapis/release-please).
Sign up for freeto subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: bigqueryIssues related to the googleapis/python-bigquery API.cla: yesThis human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

specify size parameter when uploading dataframe with load_table_from_dataframe

3 participants

@HemangChothani@tseaver@tswast