Skip to content

Core, Hive, Nessie: Use ResolvingFileIO as default instead of HadoopFileIO - #8272

Open
nastra wants to merge 1 commit into
apache:mainfrom
nastra:resolving-file-io-as-default
Open

Core, Hive, Nessie: Use ResolvingFileIO as default instead of HadoopFileIO#8272
nastra wants to merge 1 commit into
apache:mainfrom
nastra:resolving-file-io-as-default

Conversation

@nastra

Copy link
Copy Markdown
Contributor

No description provided.

@nastra
nastraforce-pushed the resolving-file-io-as-default branch from 948a917 to fb23e4dCompareAugust 9, 2023 11:10
@nastra
nastraforce-pushed the resolving-file-io-as-default branch from fb23e4d to fcbc64eCompareAugust 9, 2023 12:57

@bryanckbryanck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, with the assumption that #7976 will get merged.

@szehon-ho

Copy link
Copy Markdown
Member

Also, can we make it support SupportsPrefixOperations. Thinking it a possible option for: #8194

@nastra

Copy link
Copy Markdown
ContributorAuthor

Also, can we make it support SupportsPrefixOperations. Thinking it a possible option for: #8194

@szehon-ho did you mean to comment on #7976 w.r.t SupportsPrefixOperation rather than on this PR?

@dramaticllydramaticlly left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change LGTM, do we need to update docs anywhere to better inform user? Since this default value change and some might not be fully aware of

@nastra

Copy link
Copy Markdown
ContributorAuthor

change LGTM, do we need to update docs anywhere to better inform user? Since this default value change and some might not be fully aware of

This change should be completely transparent for users

@nastra
nastraforce-pushed the resolving-file-io-as-default branch from fcbc64e to d24c581CompareSeptember 13, 2023 15:50
private static final String NAMESPACE_EXISTS_PROPERTY = "exists";
private static final Logger LOG = LoggerFactory.getLogger(JdbcCatalog.class);
private static final Joiner SLASH = Joiner.on("/");
private static final String DEFAULT_FILE_IO_IMPL = "org.apache.iceberg.io.ResolvingFileIO";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use the name from the class potentially, rather than a string literal, as ResolvingFileIO is in core.

@rdblue

Copy link
Copy Markdown
Contributor

I think we need to discuss this more before merging it. This will change the default FileIO implementation for a lot of people -- how are we confident that it won't cause problems? Also, what is the motivation for changing the default? Are we fixing something?

@ajantha-bhat

Copy link
Copy Markdown
Member

I remember discussing this in the community sync. Why this didn't move forward?

@nastra

Copy link
Copy Markdown
ContributorAuthor

@ajantha-bhat this will be done prior to releasing 2.0.0

@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@nastra@szehon-ho@rdblue@ajantha-bhat@bryanck@dramaticlly@danielcweeks