Skip to content

ARROW-7842: [Rust] [Parquet] Arrow list reader - #8449

Closed
nevi-me wants to merge 1 commit into
apache:rust-parquet-arrow-writerfrom
nevi-me:ARROW-7842-cherry
Closed

ARROW-7842: [Rust] [Parquet] Arrow list reader#8449
nevi-me wants to merge 1 commit into
apache:rust-parquet-arrow-writerfrom
nevi-me:ARROW-7842-cherry

Conversation

@nevi-me

Copy link
Copy Markdown
Contributor

This is a port of #6770 to the parquet-writer branch.

We'll have more of a chance to test this reader,and ensure that we can roundtrip on list types.

@nevi-me
nevi-me changed the base branch from master to rust-parquet-arrow-writerOctober 12, 2020 21:46

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@carols10cents when you can, may you please help me out here on these 2 tests that I've ignored?
One needs a LargeList writer implementation, which I think you'd do better than me as you've done roundtrip for other Arrow types.

I've unignored struct_single_column as it's now passing, might be the null comparison commit that fixed it.

Thanks 😃

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at this now!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nevi-me I opened a PR on your repo with some further work -- feel free to cherry pick or whatever, but the tests aren't quite passing yet over there, just failing for a different reason ;)

@nevi-me

Copy link
Copy Markdown
ContributorAuthor

This doesn't need review, as I've addressed the questions that were in #6770. I'll wait for Carol's input before merging this.

CC @velvia@mcassels@maxburke as it was your original PR :)

@github-actions

Copy link
Copy Markdown

This is a port of apache#6770 to the parquet-writer branch.
We'll have more of a chance to test this reader,
and ensure that we can roundtrip on list types.
[Rust] [Parquet] LargeListArray support and why I think the tests are still failing (#6)
* Support reading LargeListArrays by making ListArrayReader generic over OffsetSize
* Update comment to match the actual values in this test; probably copy-paste
* Document why I think the test setup isn't quite right
disable list writer tests
They're failing because of incorrect def/rep.
Will be addressed separately
nevi-me added a commit that referenced this pull request Oct 17, 2020
This is a port of #6770 to the parquet-writer branch.
We'll have more of a chance to test this reader,and ensure that we can roundtrip on list types.
Closes#8449 from nevi-me/ARROW-7842-cherry
Authored-by: Neville Dipale <nevilledips@gmail.com>
Signed-off-by: Neville Dipale <nevilledips@gmail.com>
@nevi-me

Copy link
Copy Markdown
ContributorAuthor

Merged

@nevi-menevi-me closed this Oct 17, 2020
nevi-me added a commit that referenced this pull request Oct 25, 2020
This is a port of #6770 to the parquet-writer branch.
We'll have more of a chance to test this reader,and ensure that we can roundtrip on list types.
Closes#8449 from nevi-me/ARROW-7842-cherry
Authored-by: Neville Dipale <nevilledips@gmail.com>
Signed-off-by: Neville Dipale <nevilledips@gmail.com>
nevi-me added a commit that referenced this pull request Oct 27, 2020
This is a port of #6770 to the parquet-writer branch.
We'll have more of a chance to test this reader,and ensure that we can roundtrip on list types.
Closes#8449 from nevi-me/ARROW-7842-cherry
Authored-by: Neville Dipale <nevilledips@gmail.com>
Signed-off-by: Neville Dipale <nevilledips@gmail.com>
nevi-me added a commit that referenced this pull request Oct 28, 2020
This is a port of #6770 to the parquet-writer branch.
We'll have more of a chance to test this reader,and ensure that we can roundtrip on list types.
Closes#8449 from nevi-me/ARROW-7842-cherry
Authored-by: Neville Dipale <nevilledips@gmail.com>
Signed-off-by: Neville Dipale <nevilledips@gmail.com>
@nevi-me
nevi-me deleted the ARROW-7842-cherry branch November 7, 2020 06:28
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@nevi-me@carols10cents