Skip to content

Spark: Spark SQL read from Snapshot ref - #5294

Closed
hililiwei wants to merge 2 commits into
apache:mainfrom
hililiwei:spark_sql_use_ref
Closed

Spark: Spark SQL read from Snapshot ref#5294
hililiwei wants to merge 2 commits into
apache:mainfrom
hililiwei:spark_sql_use_ref

Conversation

@hililiwei

@hililiweihililiwei commented Jul 18, 2022

Copy link
Copy Markdown
Contributor

Reading data from a specified Ref in the following ways.

  1. USE
USE { BRANCH \ TAG } refName
  1. table path
SELECT*FROMtable.at_branch_{refName}
SELECT*FROMtable.at_branch_{refName}.at_timestamp__{timestamp}
SELECT*FROMtable.at_tag_{refName}

This is mainly in reference to our current implementation of time travel.

SELECT*FROMtable.at_timestamp__{timestamp}

It depends on #4428#5364

@hililiweihililiwei changed the title Spark: Support User Branch\Tag to read from Snapshot refSpark: Spark SQL Support User Branch\Tag to read from Snapshot refJul 18, 2022
@hililiweihililiwei changed the title Spark: Spark SQL Support User Branch\Tag to read from Snapshot refSpark: Spark SQL Support User Branch\Tag to read from Snapshot refJul 18, 2022
@hililiweihililiwei changed the title Spark: Spark SQL Support User Branch\Tag to read from Snapshot refSpark: Spark SQL Support Use Branch\Tag to read from Snapshot refJul 18, 2022
@github-actionsgithub-actionsBot removed the API label Jul 18, 2022
@hililiweihililiwei changed the title Spark: Spark SQL Support Use Branch\Tag to read from Snapshot refSpark: Spark SQL read from Snapshot refJul 27, 2022
@dungdm93

Copy link
Copy Markdown
Contributor

I prefer SELECT * FROM table@refName if it's possible

@hililiwei

Copy link
Copy Markdown
ContributorAuthor

I prefer SELECT * FROM table@refName if it's possible

Unfortunately, Spark does not support this syntax.
I could try, but not optimistic.

@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions

Copy link
Copy Markdown

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

4 participants

@hililiwei@dungdm93@jackye1995@amogh-jahagirdar