Skip to content

ARROW-10624: [R] Proactively remove "problems" attributes - #9092

Closed
jonkeane wants to merge 2 commits into
apache:masterfrom
jonkeane:r_attr
Closed

ARROW-10624: [R] Proactively remove "problems" attributes#9092
jonkeane wants to merge 2 commits into
apache:masterfrom
jonkeane:r_attr

Conversation

@jonkeane

Copy link
Copy Markdown
Member

No description provided.

@github-actions

Copy link
Copy Markdown

@nealrichardsonnealrichardson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the JIRA title to match this title (or amend both so they're the same). Also add a bugfix news bullet, let's try to keep up this week :)

Comment threadr/R/record-batch.R Outdated
Comment on lines +278 to +282
if ("attributes" %in% names(x) &&
"problems" %in% names(x[["attributes"]]) ) {
x[["attributes"]][["problems"]] <- NULL
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe you can safely just

Suggested change
if ("attributes"%in% names(x) &&
"problems"%in% names(x[["attributes"]]) ) {
x[["attributes"]][["problems"]] <-NULL
}
x[["attributes"]][["problems"]] <-NULL
> x <- list()
> x
list()
> x$attributes$problems <- NULL
> x
list()
> x[["attributes"]][["problems"]] <- NULL
> x
list()

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it turns out just NULLing doesn’t quite work with the tests for garbage metadata ("garbage"[["attributes"]][["problems"]] <- NULL errors. I can either test that the attributes are a list before this or alter our garbage tests to send in a list.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we validate/assert is.list() in this function?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I can do that. Though one of the existing tests fails here — the one right above it works which might be sufficient, is there something that lines 76-81 is testing that we aren't covering with lines 70-75?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have to look into the implementation to be sure, but I think the intent of the first test should be that we don't save garbage r metadata and the second would be that, if someone happened to have bad metadata stored in the r key (we can't prevent some other data generating process from doing that), that we don't crash on loading it. It's not clear that that's actually what those tests are doing, but that's what I think they should do.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, cool — I'll rewrite the second test to avoid using .serialize_arrow_r_metadata() so that we don't error there, but still keep the testing-serialized-data aspect of that test.

@jonkeane

Copy link
Copy Markdown
MemberAuthor

@jonkeane
jonkeane deleted the r_attr branch May 5, 2021 12:53
kou pushed a commit that referenced this pull request Feb 20, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@jonkeane@nealrichardson