Doc: Add a page explaining migration from other table formats to iceberg by JonasJ-ap · Pull Request #6600 · apache/iceberg

JonasJ-ap · 2023-01-16T00:35:18Z

Add documentation for the iceberg-delta-lake module created in #6449 and pointers to Hive/Spark table migration procedures specified in Spark Doc.

Closes#6770

jackye1995 · 2023-03-22T17:42:52Z

I think this can fix #6770 , can we update this page to a dedicated page for all migration use cases and also include Hive to Iceberg?

…tead?)

JonasJ-ap

@jackye1995 Thanks for the review. I updated the pages and made up two html preview here:

https://drive.google.com/drive/folders/1XNZ1s2rXNSUxpP_ccUBVockV5KZ3gMOG?usp=share_link
(Since github does not allow me to upload html files directly)

…t96 is in

Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>

jackye1995 · 2023-04-22T17:08:04Z

Thanks for keeping the work going on! I think we are very close, just 3 nit comments and we should be good to go! Let me know when you have another look

Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>

jackye1995

looks good to me!

jackye1995 · 2023-04-22T23:04:48Z

Thanks for all the work Jonas, merging

RussellSpitzer

Typo

RussellSpitzer · 2023-04-23T02:01:40Z

+```
+See [Spark Procedure: migrate](../spark-procedures/#migrate) for more details.
+
+## Add Files From Delta Lake Table to Iceberg


Should say hive

Oops. Thank you for catching this. Sorry for the typo.
Have opened a new PR here: #7407, to address that.

RussellSpitzer · 2023-04-23T05:58:36Z

Thanks!

…erg (apache#6600)

github-actionsBot added the docs label Jan 16, 2023

JonasJ-ap marked this pull request as ready for review January 17, 2023 03:06

JonasJ-ap mentioned this pull request Jan 24, 2023
Delta: Support Snapshot Delta Lake Table to Iceberg Table #6449
Merged

JonasJ-ap added 3 commits March 22, 2023 14:34

add doc for iceberg-delta-lake
8e6b9a7

add compatibility notice for TimestampType
65f5888

make the doc the for general table migration
198f389

JonasJ-ap force-pushed the doc_delta_lake_conversion branch from 72926c1 to 198f389Compare March 23, 2023 00:00

JonasJ-ap changed the title ~~Doc: Add descriptions and examples for Snapshot Delta Lake Table to Iceberg Table~~Doc: Add a page explaining migration from other table formats to icebergMar 23, 2023

JonasJ-ap commented Mar 23, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated
Comment threaddocs/table-migration.md Outdated

JonasJ-ap added 4 commits March 24, 2023 11:27

refactor table migration doc draft 1
d731a89

add terminal graphs
bb9ee84

adjust web pages alignment
73cc129

add pictures link (pictures should be contributed to iceberg-docs ins…
954efda
…tead?)

JonasJ-ap commented Mar 27, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 28, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

Re-organize pages and revise contents
e7f37b3

JonasJ-ap mentioned this pull request Mar 31, 2023
Add graphs for table migration section apache/iceberg-docs#220
Merged

JonasJ-ap commented Mar 31, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

jackye1995 reviewed Mar 31, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

jackye1995 reviewed Mar 31, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

JonasJ-ap added 4 commits April 6, 2023 00:02

refactor delta-lake-migration and simplify the description
732f306

Add link to CTAS, INSERT for full data migration
8960c32

add space before each section headers
da2a8c0

remove timestampType read warning since the vectorized support for in…
886930b
…t96 is in

jackye1995 reviewed Apr 19, 2023
View reviewed changes

Comment threaddocs/hive-migration.md Outdated

jackye1995 reviewed Apr 19, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

jackye1995 reviewed Apr 19, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

jackye1995 reviewed Apr 19, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

jackye1995 reviewed Apr 19, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

jackye1995 reviewed Apr 19, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

jackye1995 reviewed Apr 19, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

jackye1995 reviewed Apr 19, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

jackye1995 reviewed Apr 19, 2023
View reviewed changes

Comment threaddocs/table-migration.md Outdated

JonasJ-ap added 3 commits April 21, 2023 20:10

polish delta lake migration part
8b59815
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>

add reader/writer switching description to snapshot/migration steps
c2ee4bd
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>

revise hive migration link
7a99342
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>

jackye1995 reviewed Apr 22, 2023
View reviewed changes

Comment threaddocs/table-migration.md

jackye1995 reviewed Apr 22, 2023
View reviewed changes

Comment threaddocs/delta-lake-migration.md Outdated

JonasJ-ap added 4 commits April 22, 2023 16:36

fix code examples in delta lake migration
e2a1f37
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>

address final comments
5c31585
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>

updates link
ac462fa
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>

finalize code example
436dd2d
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>

jackye1995 approved these changes Apr 22, 2023
View reviewed changes

jackye1995 merged commit 1531e59 into apache:masterApr 22, 2023

RussellSpitzer reviewed Apr 23, 2023
View reviewed changes

JonasJ-ap mentioned this pull request Apr 23, 2023
Doc: Fix typo in hive_migration.md #7407
Merged

manisin pushed a commit to Snowflake-Labs/iceberg that referenced this pull request May 9, 2023
Doc: Add a page explaining migration from other table formats to iceb…
b697ab5
…erg (apache#6600)

Uh oh!

Conversation

JonasJ-ap commented Jan 16, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jackye1995 commented Mar 22, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JonasJ-ap left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jackye1995 commented Apr 22, 2023

Uh oh!

jackye1995 left a comment

Choose a reason for hiding this comment

Uh oh!

jackye1995 commented Apr 22, 2023

Uh oh!

RussellSpitzer left a comment

Choose a reason for hiding this comment

Uh oh!

RussellSpitzerApr 23, 2023

Choose a reason for hiding this comment

Uh oh!

JonasJ-apApr 23, 2023

Choose a reason for hiding this comment

Uh oh!

RussellSpitzer commented Apr 23, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JonasJ-ap commented Jan 16, 2023•
edited
Loading

jackye1995 commented Mar 22, 2023•
edited
Loading