Uh oh!
There was an error while loading. Please reload this page.
[AIRFLOW-5889] Fix polling for AWS Batch job status - #6765
Merged
Conversation
dazza-codesforce-pushed
the
aws-batch-poll-exceptions
branch
2 times, most recently
from
December 10, 2019 05:52
7c29b4d to
7f65c55CompareCodecov Report
@@ Coverage Diff @@## master #6765 +/- ##
=========================================
- Coverage 84.32% 84.3% -0.02%
=========================================
Files 672 672 Lines 38179 38210 +31 =========================================
+ Hits 32195 32214 +19 - Misses 5984 5996 +12
Continue to review full report at Codecov.
|
ContributorAuthor
The bug fix in this PR should get into a 1.x release, so it might be better if this PR sneaks in before the AIP-21 stuff in #6764 is resolved. |
ashb
reviewed
Dec 11, 2019
Uh oh!
There was an error while loading. Please reload this page.
ashb
reviewed
Dec 11, 2019
ashb
left a comment
Member
There was a problem hiding this comment.
LGTM - one small change requested, then we can merge this.
dimberman
previously requested changes
Dec 11, 2019
Uh oh!
There was an error while loading. Please reload this page.
- errors in polling for job status should not fail
the airflow task when the polling hits an API throttle
limit; polling should detect those cases and retry a
few times to get the job status, only failing the task
when the job description cannot be retrieved
- added typing for the BatchProtocol method return
types, based on the botocore.client.Batch types
- applied trivial format consistency using black, i.e.
$ black -t py36 -l 96 {files}dazza-codesforce-pushed
the
aws-batch-poll-exceptions
branch
from
December 12, 2019 00:37
7f65c55 to
c52463eCompare5 tasks
kaxil pushed a commit
that referenced
this pull request
Dec 17, 2019
…6765) - errors in polling for job status should not fail the airflow task when the polling hits an API throttle limit; polling should detect those cases and retry a few times to get the job status, only failing the task when the job description cannot be retrieved - added typing for the BatchProtocol method return types, based on the botocore.client.Batch types (NOT INCLUDED IN THE CHERRY PICK) - applied trivial format consistency using black, i.e. $ black -t py36 -l 96 {files} (cherry picked from commit 479ee63)
5 tasks
kaxil pushed a commit
that referenced
this pull request
Dec 18, 2019
…6765) - errors in polling for job status should not fail the airflow task when the polling hits an API throttle limit; polling should detect those cases and retry a few times to get the job status, only failing the task when the job description cannot be retrieved - added typing for the BatchProtocol method return types, based on the botocore.client.Batch types (NOT INCLUDED IN THE CHERRY PICK) - applied trivial format consistency using black, i.e. $ black -t py36 -l 96 {files} (cherry picked from commit 479ee63)
ashb pushed a commit
that referenced
this pull request
Dec 19, 2019
…6765) - errors in polling for job status should not fail the airflow task when the polling hits an API throttle limit; polling should detect those cases and retry a few times to get the job status, only failing the task when the job description cannot be retrieved - added typing for the BatchProtocol method return types, based on the botocore.client.Batch types (NOT INCLUDED IN THE CHERRY PICK) - applied trivial format consistency using black, i.e. $ black -t py36 -l 96 {files} (cherry picked from commit 479ee63)
galuszkak pushed a commit
to FlyrInc/apache-airflow
that referenced
this pull request
Mar 5, 2020
…pache#6765) - errors in polling for job status should not fail the airflow task when the polling hits an API throttle limit; polling should detect those cases and retry a few times to get the job status, only failing the task when the job description cannot be retrieved - added typing for the BatchProtocol method return types, based on the botocore.client.Batch types - applied trivial format consistency using black, i.e. $ black -t py36 -l 96 {files}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR target
This PR targets a 1.10.x release for this bug fix
1.10.testbranch instead ofmaster?Jira
Possible conflicts
Description
the airflow task when the polling hits an API throttle
limit; polling should detect those cases and retry a
few times to get the job status
types, based on the botocore.client.Batch types
$ black -t py36 -l 96 {files}
Tests
Commits
Documentation