Skip to content

Normalize handling 403 for rate limits on 'dataset.create()'. - #2089

Merged
tseaver merged 3 commits into
googleapis:masterfrom
tseaver:2075-bigquery-flaky-system-tests
Aug 11, 2016
Merged

Normalize handling 403 for rate limits on 'dataset.create()'.#2089
tseaver merged 3 commits into
googleapis:masterfrom
tseaver:2075-bigquery-flaky-system-tests

Conversation

@tseaver

@tseavertseaver commented Aug 11, 2016

Copy link
Copy Markdown
Contributor

Fixes#2075.

@tseavertseaver added testing api: bigquery Issues related to the BigQuery API. labels Aug 11, 2016
@googlebotgooglebot added the cla: yes This human has signed the Contributor License Agreement. label Aug 11, 2016
dhermes
dhermes reviewed Aug 11, 2016
View reviewed changes
Comment threadsystem_tests/bigquery.py

# We need to wait to stay within the rate limits.
# The alternative outcome is a 403 Forbidden response from upstream.
# See: https://cloud.google.com/bigquery/quota-policy

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@tseaver
Explain back-end's use of 403 for rate limiting.
7dfa1ad
A 429 ("Too many requests") would be way more appropriate than a 403
("Forbidden"), but there is nothing we can do about it here.
Addresses:
#2089 (comment)
@dhermes

dhermes commented Aug 11, 2016

Copy link
Copy Markdown
Contributor

Re: the error predicate -- I'm reluctant to add it because it makes the test flakier with respect to the back-end, which might change the error text arbitrarily and without notice.

We could deal with backend changes if / when tests fail. You can make it a bit less brittle via:

'rate limit'inerror.message.lower()

@tseaver

tseaver commented Aug 11, 2016•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

In 10e4f51, I figured out that we capture the JSON error structure, which means we can use the documentedrateLimitExceeded reason code, rather than sniffing at the message text.

@tseaver
Add error predicate to 'retry_403'.
10e4f51
Exclude Forbidden exceptions which do not have 'rateLimitExceed' as the
reason for one of their errors.
Addresses:
#2089 (comment).
@dhermes

dhermes commented Aug 11, 2016

Copy link
Copy Markdown
Contributor

Nice find! LGTM

@tseaver
tseaver merged commit a99f269 into googleapis:masterAug 11, 2016
@tseaver
tseaver deleted the 2075-bigquery-flaky-system-tests branch August 11, 2016 21:09
@dhermesdhermes mentioned this pull request Sep 19, 2016
Merged
parthea pushed a commit that referenced this pull request Apr 1, 2026
@GarrettWu
fix: Transformers with non-standard column names throw errors (#2089)
055dded
* fix: Transformers with non-standard column names through errors
* fix
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

api: bigqueryIssues related to the BigQuery API.cla: yesThis human has signed the Contributor License Agreement.testing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tseaver@dhermes@googlebot