Skip to content

[View Support] Add ViewMetadata read support and create_view to REST Catalog - #2154

Merged
Fokko merged 22 commits into
apache:mainfrom
rambleraptor:create_view
Mar 7, 2026
Merged

[View Support] Add ViewMetadata read support and create_view to REST Catalog#2154
Fokko merged 22 commits into
apache:mainfrom
rambleraptor:create_view

Conversation

@rambleraptor

Copy link
Copy Markdown
Collaborator

Part of #818

This adds create_view to the REST Catalog. As part of this, it also defines the View and ViewMetadata class.

Rationale for this change

PyIceberg's REST Catalog doesn't support create_view. Furthermore, PyIceberg doesn't support views at all. This is the first part in getting views supported.

Are these changes tested?

Unit tests included

Are there any user-facing changes?

Added Iceberg REST Catalog support for create_view

@ndrluisndrluis mentioned this pull request Jun 27, 2025
8 tasks
Comment threadpyiceberg/view/metadata.py Outdated
Comment threadpyiceberg/view/metadata.py
@jayceslesar

Copy link
Copy Markdown
Contributor

Is it worth setting integration tests in rest integration tests?

@rambleraptor

Copy link
Copy Markdown
CollaboratorAuthor

I'm a little unclear what Catalog the REST integration tests run against, so I can't say if there's been enough implemented to actually have a valid REST Catalog test.

The integration tests are missing most of the methods that have been implemented, so my reaction is that we're still missing pieces for an integration test.

@rambleraptor

Copy link
Copy Markdown
CollaboratorAuthor

@Fokko@kevinjqliu mind taking a look when you can? thanks!

Comment threadtests/conftest.py

@sungwysungwy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this up @rambleraptor

I've taken a stab at reviewing the PR. I think adding an integration test is a must-have for introducing a new API. I think a great way to verify that the created view works as expected is to add a view as a spark dialect sql view and read the view table through a spark session, similar to how we validate that PyIceberg writes are reflected correctly even when the table is read through a Spark session accessing the same REST Catalog (example). WDYT?

Comment threadpyiceberg/view/metadata.py Outdated
Comment threadpyiceberg/view/metadata.py
Comment threadpyiceberg/view/metadata.py Outdated
Comment threadpyiceberg/view/metadata.py Outdated
Comment threadpyiceberg/catalog/rest/__init__.py Outdated
Comment threadpyiceberg/view/metadata.py Outdated
@rambleraptor

Copy link
Copy Markdown
CollaboratorAuthor

@sungwy@jayceslesar please take a look when you can!

Comment threadpyiceberg/view/__init__.py Outdated
Comment threadpyiceberg/view/metadata.py Outdated
@rambleraptor
rambleraptor requested a review from FokkoJuly 3, 2025 00:17
@rambleraptor

Copy link
Copy Markdown
CollaboratorAuthor

addressed the comments, thanks for the review!

@sungwy

Copy link
Copy Markdown
Collaborator

Hi @rambleraptor the CI is failing due to a lint error.

pyiceberg/view/__init__.py:23:22: F401 `pydantic.Field` imported but unused
|
21 | )
22 |
23 | from pydantic import Field
| ^^^^^ F401
24 |
25 | from pyiceberg.typedef import Identifier
|
= help: Remove unused import: `pydantic.Field`

Could you run make lint and commit the changes?

@rambleraptor

Copy link
Copy Markdown
CollaboratorAuthor

@sungwy whoops, thanks! all fixed.

@rambleraptor

Copy link
Copy Markdown
CollaboratorAuthor

@sungwy@jayceslesar@Fokko can I get a review on this when you can? Thanks!

@rambleraptorrambleraptor changed the title Add create_view to REST Catalog[View Support] Add create_view to REST CatalogJul 24, 2025
@rambleraptorrambleraptor changed the title [View Support] Add create_view to REST Catalog[View Support] Add ViewMetadata read support and create_view to REST CatalogJul 25, 2025
@sjdurfey

Copy link
Copy Markdown

is this issue dead?

@rambleraptor

Copy link
Copy Markdown
CollaboratorAuthor

@sjdurfey I just updated the PR to handle the merge conflicts. I'm definitely interested in getting more View support in and I think this PR is a good place to begin!

@kevinjqliu@Fokko fyi, ptal!

@FokkoFokko left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some small comments, but this looks good to me 👍

Comment threadpyiceberg/catalog/bigquery_metastore.py Outdated
Comment threadpyiceberg/view/metadata.py Outdated
rambleraptorand others added 17 commits February 23, 2026 10:14
Co-authored-by: Sung Yun <107272191+sungwy@users.noreply.github.com>
Co-authored-by: Sung Yun <107272191+sungwy@users.noreply.github.com>
Co-authored-by: Sung Yun <107272191+sungwy@users.noreply.github.com>
Co-authored-by: Sung Yun <107272191+sungwy@users.noreply.github.com>
Co-authored-by: Sung Yun <107272191+sungwy@users.noreply.github.com>
@rambleraptor

Copy link
Copy Markdown
CollaboratorAuthor

@Fokko changes made! On vacation the last week, thanks for being patient!

@Fokko
Fokko merged commit c5c0993 into apache:mainMar 7, 2026
14 checks passed
@Fokko

Fokko commented Mar 7, 2026

Copy link
Copy Markdown
Contributor

Hope you had a great vacation @rambleraptor, and thanks for working on this!

kris-gaudel pushed a commit to kris-gaudel/iceberg-python that referenced this pull request Mar 13, 2026
…he#2154)
<!--
Thanks for opening a pull request!
-->
<!-- In the case this PR will resolve an issue, please replace
${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->
Part of apache#818 This adds `create_view` to the REST Catalog. As part of this, it also
defines the View and ViewMetadata class.
# Rationale for this change
PyIceberg's REST Catalog doesn't support `create_view`. Furthermore,
PyIceberg doesn't support views at all. This is the first part in
getting views supported.
# Are these changes tested?
Unit tests included
# Are there any user-facing changes?
Added Iceberg REST Catalog support for `create_view`
<!-- In the case of user-facing changes, please add the changelog label.
-->
---------
Co-authored-by: Sung Yun <107272191+sungwy@users.noreply.github.com>
Co-authored-by: Fokko Driesprong <fokko@apache.org>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@rambleraptor@jayceslesar@sungwy@sjdurfey@Fokko