Skip to content

Support custom KMS keys in S3CopyObjectOperator - #60597

Merged
shahar1 merged 7 commits into
apache:mainfrom
dominikhei:s3-copy-kms-support
Feb 10, 2026
Merged

Support custom KMS keys in S3CopyObjectOperator#60597
shahar1 merged 7 commits into
apache:mainfrom
dominikhei:s3-copy-kms-support

Conversation

@dominikhei

Copy link
Copy Markdown
Contributor

closes: #55708

This PR introduces 2 new parameters to the S3CopyObjectOperator. With these one can specify a non-default KMS key when copying objects between buckets.

  • kms_key_id: The ARN, ID or alias of a KMS key
  • kms_encryption_type: Whether it is standard KMS or double-shielded KMS

The parameters are passed to the Hooks copy_object() method using kwargs, which passes them to the boto3 method.
I did this to not introduce new parameters to the method and keep changes minimal, however I also see a point that this is not as clean as passing them to the method via parameters and if deemed preferential will change that.


Was generative AI tooling used to co-author this PR?
  • [] Yes

@boring-cyborgboring-cyborgBot added area:providers provider:amazon AWS/Amazon - related issues labels Jan 15, 2026
@eladkal
eladkal requested a review from vincbeckJanuary 15, 2026 16:21

@o-nikolaso-nikolas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good other than the two comments already called out.

You also might want to update the docs to describe how/when to use these, might be helpful for some folks

Comment threadproviders/amazon/src/airflow/providers/amazon/aws/hooks/s3.py Outdated
Comment threadproviders/amazon/src/airflow/providers/amazon/aws/hooks/s3.py Outdated
Comment threadproviders/amazon/tests/unit/amazon/aws/hooks/test_s3.py Outdated
@shahar1

Copy link
Copy Markdown
Contributor

@dominikhei If there aren't critical changes to make, please let the CI run E2E without merging from main - so I could include it in the release I cut in the next hour or so. Thank you!

@shahar1shahar1 changed the title S3 Copy Operator: Support Custom KMS KeysSupport custom KMS Keys in S3CopyObjectOperatorFeb 10, 2026
@shahar1
shahar1 merged commit fdb207e into apache:mainFeb 10, 2026
90 checks passed
@shahar1shahar1 changed the title Support custom KMS Keys in S3CopyObjectOperatorSupport custom KMS keys in S3CopyObjectOperatorFeb 10, 2026
Alok-kumar-priyadarshi pushed a commit to Alok-kumar-priyadarshi/airflow that referenced this pull request Feb 11, 2026
Ratasa143 pushed a commit to Ratasa143/airflow that referenced this pull request Feb 15, 2026
choo121600 pushed a commit to choo121600/airflow that referenced this pull request Feb 22, 2026
AkshayArali pushed a commit to AkshayArali/airflow_630 that referenced this pull request Feb 27, 2026
AkshayArali pushed a commit to AkshayArali/airflow_630 that referenced this pull request Feb 27, 2026
Subham-KRLX pushed a commit to Subham-KRLX/airflow that referenced this pull request Mar 4, 2026
dominikhei added a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
radhwene pushed a commit to radhwene/airflow that referenced this pull request Mar 21, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providersprovider:amazonAWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for passing KMS key when copying S3 objects across AWS accounts using S3CopyObjectOperator

5 participants

@dominikhei@shahar1@o-nikolas@SameerMesiah97@vincbeck