Skip to content

ARROW-3761: [R] Bindings for CompressedInputStream, CompressedOutputStream - #2962

Closed
romainfrancois wants to merge 8 commits into
apache:masterfrom
romainfrancois:ARROW-3761/CompressedStream
Closed

ARROW-3761: [R] Bindings for CompressedInputStream, CompressedOutputStream#2962
romainfrancois wants to merge 8 commits into
apache:masterfrom
romainfrancois:ARROW-3761/CompressedStream

Conversation

@romainfrancois

@romainfrancoisromainfrancois commented Nov 14, 2018

Copy link
Copy Markdown
Contributor

No description provided.

@codecov-io

codecov-io commented Nov 14, 2018

Copy link
Copy Markdown

Codecov Report

Merging #2962 into master will increase coverage by 15.14%.
The diff coverage is n/a.

Impacted file tree graph

@@ Coverage Diff @@## master #2962 +/- ##
===========================================
+ Coverage 74.31% 89.45% +15.14% 
===========================================
Files 68 12 -56 Lines 5423 1811 -3612 ===========================================
- Hits 4030 1620 -2410 + Misses 1297 191 -1106 + Partials 96 0 -96
Impacted FilesCoverage Δ
go/arrow/array/table.go
go/arrow/math/uint64_amd64.go
go/arrow/internal/testing/tools/bool.go
go/arrow/internal/bitutil/bitutil.go
go/arrow/memory/memory_avx2_amd64.go
go/arrow/array/null.go
go/arrow/datatype_nested.go
go/arrow/array/string.go
go/arrow/math/uint64_avx2_amd64.go
go/arrow/array/builder.go
... and 46 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 60305d5...8ccc977. Read the comment docs.

@romainfrancois
romainfrancoisforce-pushed the ARROW-3761/CompressedStream branch 3 times, most recently from 8ccc977 to 633d1ccCompareNovember 19, 2018 11:19
@romainfrancois
romainfrancoisforce-pushed the ARROW-3761/CompressedStream branch from 633d1cc to 1013a1dCompareNovember 26, 2018 09:48
@wesm

wesm commented Nov 26, 2018

Copy link
Copy Markdown
Member

Are you working on this somewhere else?

@romainfrancois

Copy link
Copy Markdown
ContributorAuthor

I'm not.

@romainfrancois

Copy link
Copy Markdown
ContributorAuthor

I don't know what happened, but it looks I've lost some commits in that branch. this is thankfully minor.

@wesm

wesm commented Nov 26, 2018

Copy link
Copy Markdown
Member

Looks like you may need to git reset --hard to 8ccc977 and then rebase. It should be in your reflog

@wesmwesm reopened this Nov 26, 2018
@romainfrancois

Copy link
Copy Markdown
ContributorAuthor

Thanks @wesm

@wesm

wesm commented Nov 28, 2018

Copy link
Copy Markdown
Member

There are no tests here. What would you like to do?

@romainfrancois

Copy link
Copy Markdown
ContributorAuthor

I’m still unsure how to test this. I’ll look around in pyarrow next time i look at this branch

@wesm

wesm commented Nov 28, 2018

Copy link
Copy Markdown
Member

Got it. I think we just generate random bytes and write them to a compressed file, then read them back in and compare

@romainfrancois

Copy link
Copy Markdown
ContributorAuthor

Makes sense. IIRC I tried to serialize a record batch but that did not work out well.

@wesm

wesm commented Nov 28, 2018

Copy link
Copy Markdown
Member

Yeah, that requires random access. It should at least not crash, though

@wesm

wesm commented Nov 28, 2018

Copy link
Copy Markdown
Member

Well, to read using the *StreamReader should work. If it does not, that's buggy

@romainfrancois

Copy link
Copy Markdown
ContributorAuthor

Thanks. I’ll use that in testing.

@romainfrancois
romainfrancoisforce-pushed the ARROW-3761/CompressedStream branch from 6002218 to efe4ae8CompareDecember 3, 2018 08:43
@romainfrancois

Copy link
Copy Markdown
ContributorAuthor

This will likely need to be rebased once #3043 is merged

@romainfrancois
romainfrancoisforce-pushed the ARROW-3761/CompressedStream branch from 0ba65a0 to 14a7ef1CompareDecember 4, 2018 20:33
@romainfrancois
romainfrancoisforce-pushed the ARROW-3761/CompressedStream branch 2 times, most recently from cd5e8c9 to 6c8907cCompareDecember 10, 2018 18:15
@romainfrancois
romainfrancoisforce-pushed the ARROW-3761/CompressedStream branch from 6c8907c to feb948aCompareJanuary 2, 2019 09:42
@romainfrancois
romainfrancoisforce-pushed the ARROW-3761/CompressedStream branch from feb948a to 397bacaCompareJanuary 7, 2019 15:19

@wesmwesm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@wesmwesm closed this in f576c3eJan 29, 2019
@romainfrancois
romainfrancois deleted the ARROW-3761/CompressedStream branch February 5, 2019 09:32
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@romainfrancois@codecov-io@wesm