Uh oh!
There was an error while loading. Please reload this page.
ARROW-10386 [R]: List column class attributes not preserved in roundtrip - #8549
ARROW-10386 [R]: List column class attributes not preserved in roundtrip#8549romainfrancois wants to merge 2 commits into
Conversation
romainfrancois
commented
Oct 29, 2020
This increases again the weight of the metadata, which now has to include attributes for each element of a list column, aka a struct array, but this seems on this example to work. Can't use test_that("metadata of list elements (ARROW-10386)", {
df<-data.frame(x= I(list(structure(1, foo="bar"), structure(2, foo="bar"))))
tab<-Table$create(df)
expect_identical(attr(as.data.frame(tab)$x[[1]], "foo"), "bar")
expect_identical(attr(as.data.frame(tab)$x[[2]], "foo"), "bar")
}) |
petrbouchal
commented
Nov 1, 2020
Thanks - can confirm the sf object now roundtrips correctly on my machine as well using the current HEAD version. |
…st itself. ARROW-10386.
d716a74 to
4d1e73dCompare
jonkeane
left a comment
There was a problem hiding this comment.
This looks good to me, a few minor suggestions
| apply_arrow_r_metadata(.x, .y) | ||
| }) | ||
| x | ||
| } |
There was a problem hiding this comment.
Could we move !is.null(columns_metadata) up to line 290 instead of having it on both 291 and 296?
| df <- data.frame(x = I(list(structure(1, foo = "bar"), structure(2, foo = "bar")))) | ||
| tab <- Table$create(df) | ||
| expect_identical(attr(as.data.frame(tab)$x[[1]], "foo"), "bar") | ||
| expect_identical(attr(as.data.frame(tab)$x[[2]], "foo"), "bar") |
There was a problem hiding this comment.
I wonder if it would be clearer to have different attributes on each item/row to make it super obvious that we're not picking the attributes of the first item/row and copying them for the whole column.
nealrichardson
commented
Jan 4, 2021
How much does it blow up the metadata? Is this going to scale to be able to handle normal/large shapefiles? Is there a more efficient representation for this metadata (considering that we have to serialize it to a string)? Should we special-case |
nealrichardson
commented
Jan 13, 2021
Closing in favor of #9182; relevant commits cherry-picked to there. |
…Hub issue numbers (#34260) Rewrite the Jira issue numbers to the GitHub issue numbers, so that the GitHub issue numbers are automatically linked to the issues by pkgdown's auto-linking feature. Issue numbers have been rewritten based on the following correspondence. Also, the pkgdown settings have been changed and updated to link to GitHub. I generated the Changelog page using the `pkgdown::build_news()` function and verified that the links work correctly. --- ARROW-6338#5198ARROW-6364#5201ARROW-6323#5169ARROW-6278#5141ARROW-6360#5329ARROW-6533#5450ARROW-6348#5223ARROW-6337#5399ARROW-10850#9128ARROW-10624#9092ARROW-10386#8549ARROW-6994#23308ARROW-12774#10320ARROW-12670#10287ARROW-16828#13484ARROW-14989#13482ARROW-16977#13514ARROW-13404#10999ARROW-16887#13601ARROW-15906#13206ARROW-15280#13171ARROW-16144#13183ARROW-16511#13105ARROW-16085#13088ARROW-16715#13555ARROW-16268#13550ARROW-16700#13518ARROW-16807#13583ARROW-16871#13517ARROW-16415#13190ARROW-14821#12154ARROW-16439#13174ARROW-16394#13118ARROW-16516#13163ARROW-16395#13627ARROW-14848#12589ARROW-16407#13196ARROW-16653#13506ARROW-14575#13160ARROW-15271#13170ARROW-16703#13650ARROW-16444#13397ARROW-15016#13541ARROW-16776#13563ARROW-15622#13090ARROW-18131#14484ARROW-18305#14581ARROW-18285#14615 * Closes: #33631 Authored-by: SHIMA Tatsuya <ts1s1andn@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
On the original example:
Created on 2020-10-29 by the reprex package (v0.3.0.9001)