Skip to content

Add parameter to pass role ARN to GlueJobOperator - #33408

Merged
vincbeck merged 10 commits into
apache:mainfrom
erdos2n:main
Aug 15, 2023
Merged

Add parameter to pass role ARN to GlueJobOperator #33408
vincbeck merged 10 commits into
apache:mainfrom
erdos2n:main

Conversation

@erdos2n

@erdos2nerdos2n commented Aug 15, 2023•
edited by vincbeck
Loading

Copy link
Copy Markdown
Contributor

GlueOperator currently requires iam_role_name and endpoint_url to fetch the ARN from AWS. This ARN eventually gets passed into the config for Glue Job.

Resolves#33248

@boring-cyborgboring-cyborgBot added area:providers provider:amazon AWS/Amazon - related issues labels Aug 15, 2023
@erdos2n

erdos2n commented Aug 15, 2023•
edited
Loading

Copy link
Copy Markdown
ContributorAuthor

@eladkal , @o-nikolas I added test cases for the role_arn, can you let me know if the tests were added correctly?
Thank you

vincbeck
vincbeck reviewed Aug 15, 2023
View reviewed changes
Comment threadairflow/providers/amazon/aws/hooks/glue.py Outdated
Comment threadairflow/providers/amazon/aws/hooks/glue.py Outdated
Comment threadairflow/providers/amazon/aws/hooks/glue.py Outdated
Comment threadairflow/providers/amazon/aws/operators/glue.py Outdated
Comment threadtests/providers/amazon/aws/hooks/test_glue.py
erdos2n
erdos2n commented Aug 15, 2023
View reviewed changes
Comment threadtests/providers/amazon/aws/hooks/test_glue.py
vincbeck
vincbeck approved these changes Aug 15, 2023
View reviewed changes

@vincbeckvincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vincbeck

vincbeck commented Aug 15, 2023

Copy link
Copy Markdown
Contributor

You have some static checks failure, please run them to fix these failures.

@erdos2nerdos2n changed the title [WIP] Issue ##33248 - GlueOperator IAM ARNIssue ##33248 - GlueOperator IAM ARNAug 15, 2023
@vincbeckvincbeck changed the title Issue ##33248 - GlueOperator IAM ARNAdd parameter to pass role ARN to GlueJobOperator Aug 15, 2023
@vincbeck
vincbeck merged commit 60df705 into apache:mainAug 15, 2023
ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 17, 2023
@erdos2n@ferruzzi
Add parameter to pass role ARN toGlueJobOperator(apache#33408)
6cecf08
@eladkaleladkal mentioned this pull request Aug 25, 2023
Closed
63 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vincbeckvincbeckvincbeck approved these changes
@eladkaleladkalAwaiting requested review from eladkal
@o-nikolaso-nikolasAwaiting requested review from o-nikolas

Assignees

No one assigned

Labels

area:providersprovider:amazonAWS/Amazon - related issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

GlueOperator: iam_role_arn as a parameter

2 participants

@erdos2n@vincbeck