Skip to content

Core: Pass storage credentials from LoadTableResponse to FileIO - #12591

Merged
nastra merged 3 commits into
apache:mainfrom
nastra:creds-from-loadtable
Apr 2, 2025
Merged

Core: Pass storage credentials from LoadTableResponse to FileIO#12591
nastra merged 3 commits into
apache:mainfrom
nastra:creds-from-loadtable

Conversation

@nastra

Copy link
Copy Markdown
Contributor

No description provided.

@nastra
nastra marked this pull request as draft March 20, 2025 18:16
@nastra
nastraforce-pushed the creds-from-loadtable branch from 9bc2853 to 8db24dfCompareMarch 20, 2025 18:32
Comment threadcore/src/main/java/org/apache/iceberg/rest/responses/LoadTableResponse.java Outdated
@nastra
nastraforce-pushed the creds-from-loadtable branch from 8db24df to efe6592CompareMarch 27, 2025 16:29
@nastra
nastraforce-pushed the creds-from-loadtable branch from efe6592 to 716a064CompareMarch 27, 2025 16:31
@nastranastra changed the title Core: Use credentials from LoadTableResponse if availableCore: Pass storage credentials from LoadTableResponse to FileIOMar 27, 2025
@nastra
nastraforce-pushed the creds-from-loadtable branch 6 times, most recently from 7848acb to 0c21c44CompareMarch 28, 2025 14:56
@nastra
nastra marked this pull request as ready for review March 28, 2025 14:57
@nastra
nastraforce-pushed the creds-from-loadtable branch 4 times, most recently from 2f64867 to 0ca4c6bCompareMarch 31, 2025 14:43

@amogh-jahagirdaramogh-jahagirdar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nastra! Passing through the credentials to the fileIO implementations looks great. Just had a comment for adding a JavaDoc on the new mixin interface since it is public.

Comment threadcore/src/main/java/org/apache/iceberg/rest/responses/LoadTableResponse.java Outdated
@nastra
nastraforce-pushed the creds-from-loadtable branch from 0ca4c6b to 7d75581CompareApril 2, 2025 06:11
@nastra

Copy link
Copy Markdown
ContributorAuthor

thanks for the reviews @amogh-jahagirdar and @danielcweeks

@nastra
nastra merged commit 817dc35 into apache:mainApr 2, 2025
@nastra
nastra deleted the creds-from-loadtable branch April 2, 2025 09:55
}

@Override
public void setCredentials(List<StorageCredential> credentials) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I observe, the this.storageCredentials can be wrapped in ImmutableList.copyOf. This way, credentials() doesn't have to create a copy every time.

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will not work with Kryo ser/de because Kryo will try to add elements back after deserialization into the list and then fail

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the getter can be written this way (with additional field called copy):
first time getter assigns ImmutableList.copyOf() to copy field

subsequently copy field is returned directly until setter is called

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@nastra@tedyu@danielcweeks@amogh-jahagirdar