ARROW-16776: [R] dplyr::glimpse method for arrow table and datasets

[nealrichardson]

See reprex (sans terminal formatting) in r/tests/testthat/_snaps/dplyr-glimpse.md

Not all queries can be glimpse()d: some would require evaluating the whole query, which may be expensive (and can't be interrupted yet, see ARROW-11841).

Note that the existing print() methods aren't affected by this. There is still the idea that the print methods for Table/RecordBatch should print some data (ARROW-16777 and others), but that should probably be column-oriented instead of row-oriented like glimpse().

[github-actions]

https://issues.apache.org/jira/browse/ARROW-16776

[wjones127]

Question: Is there a reason we didn't want the utils::?

[nealrichardson]

We've elsewhere done importFrom so it's not necessary

[nealrichardson]

@jthomasmock FYI

[jthomasmock]

@jthomasmock FYI

Very excited to test this out once it's in dev, had peeked through the testthat outputs/snapshots!

[wjones127]

This looks great. I tried to throw something a little more complex and it did well at formatting the output:

library(arrow)
library(dplyr)
tab<- arrow_table(
x=Array$create(c(1, 2, 3)),
extremely_long_name_of_a_column_here=Array$create(list(
list(data.frame(x= rep("XXXXXXXXXXXXXXXX", 100))),
list(data.frame(x= rep("YYYYYY", 100))),
list(data.frame(x= rep("ZZZZZZZZZ", 100)))
))
)
glimpse(tab)
#> Table#> 3 rows x 2 columns#> $ x <double> 1, 2, 3#> $ extremely_long_name_of_a_column_here <list<...>> [[<tbl_df[100 x 1]>]], [[<tbl…#> Call `print()` for full schema details

^{Created on 2022-07-11 by the reprex package (v2.0.1)}

[wjones127]

The reason we can't push this down to C++ is because we haven't constructed an exec plan yet, right? Otherwise, it would be more maintainable to do so.

[nealrichardson]

I don't follow. We could build an ExecPlan, but it wouldn't tell us anything about how it would perform, would it? I'm trying to detect cases where I can just take head() of the data without having to scan an entire dataset.

[wjones127]

We could build an ExecPlan, but it wouldn't tell us anything about how it would perform, would it?

I'm not super close to the ExecPlan code, but I thought they were composed of a graph of nodes that could be traversed and analyzed, just like our arrow_dplyr_query structure. Am I wrong on that?

I'm trying to detect cases where I can just take head() of the data without having to scan an entire dataset.

I was just thinking that having such a method on ExecPlan would be useful in general.

[nealrichardson]

Sure, that probably would be useful

[westonpace]

I was just thinking that having such a method on ExecPlan would be useful in general.

Possibly. We'd probably want to define it more formally. SQL has LIMIT X and Substrait's equivalent is FetchRel. Neither of these are exactly what is being detected here. For example, it is legal to have SELECT SUM(x) FROM table LIMIT 1 but it wouldn't actually limit any data being read.

We could define it as "single pipeline queries" but a pipeline breaker doesn't necessarily mean a query is non-streaming (for example, hash-join is sometimes permitted as "streaming" in this example but it is always a pipeline breaker).

[nealrichardson]

Since you mentioned limit, I'll make a plug for ARROW-16628. Not relevant for this particular question, just would let me delete some R specific handling outside of the ExecPlan, and I'm guessing we'll have to do it to support substrait.