[R] stringr modifier functions cannot be called with namespace prefix

[giocomai]

Describe the bug, including details regarding any error messages, version, and platform.

When using stringr's str_detect() and str_count(), stringr's own documentation recommends to use stringr::regex() and stringr::fixed() "for finer control of the matching behaviour."

This can be used, for example, to set "ignore_case" to TRUE, which is not available as an argument to str_detect() directly.

The resulting functions have the following structure:

stringr::str_detect(
 string = "eXample",
 pattern = stringr::regex("x", ignore_case = TRUE)
)
#> [1] TRUE

Unfortunately, arguments passed via stringr::regex() and stringr::fixed() are silently ignored by arrow, which leads to unexpected and quite possibly wrong results.

If one prints the arrow call, it is possible to see that indeed even if ignore_case is set to TRUE, the call is passed with ignore_case as FALSE.

bool (match_substring_regex(text, {pattern="x", ignore_case=false}))

I suppose arrow should either get this right, or throw an error.

The following reprex (run with arrow version 12.0.1) shows:

• how the ignore.case argument works nicely when passed via the base function grepl
• how it is simply ignored when passed to stringr::str_detect(), stringr::str_count() (and possibly other stringr functions) through stringr::regex() and stringr::str_detect()
• how it works nicely if the ignore_case is passed directly in the pattern with (?i)
• how arrow throws an error when using stringi::stri_detect_regex() (rather than stringr) with case_insensitive = TRUE (which is still preferrable to ignoring the argument silently).

There are obviously many workarounds, but this has led to errors when I applied functions that were not originally written and tested with arrow in mind.

library("arrow")
#> 
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#> 
#> timestamp

apple_df <- tibble::tibble(
 text = c(
 "apple",
 "APPLE"
 )
)

arrow::write_dataset(dataset = apple_df, path = "apple.parquet")

apple_parquet <- arrow::open_dataset(sources = "apple.parquet")



## with grepl, it works

apple_parquet |>
 dplyr::mutate(
 a_check = grepl(
 x = text,
 pattern = "a",
 ignore.case = TRUE
 )
 )
#> FileSystemDataset (query)
#> text: string
#> a_check: bool (if_else(is_null(match_substring_regex(text, {pattern="a", ignore_case=true}), {nan_is_null=true}), false, match_substring_regex(text, {pattern="a", ignore_case=true})))
#> 
#> See $.data for the source Arrow object

apple_parquet |>
 dplyr::mutate(
 a_check = grepl(x = text, pattern = "a", ignore.case = TRUE)
 ) |>
 dplyr::collect()
#> # A tibble: 2 × 2
#> text a_check
#> <chr> <lgl> 
#> 1 apple TRUE 
#> 2 APPLE TRUE


## with stringr::str_detect it does not work

apple_parquet |>
 dplyr::mutate(
 a_check = stringr::str_detect(
 string = text,
 pattern = "a"
 )
 )
#> FileSystemDataset (query)
#> text: string
#> a_check: bool (match_substring_regex(text, {pattern="a", ignore_case=false}))
#> 
#> See $.data for the source Arrow object


apple_parquet |>
 dplyr::mutate(
 a_check = stringr::str_detect(
 string = text,
 pattern = stringr::regex(
 pattern = "a",
 ignore_case = TRUE
 )
 )
 )
#> FileSystemDataset (query)
#> text: string
#> a_check: bool (match_substring_regex(text, {pattern="a", ignore_case=false}))
#> 
#> See $.data for the source Arrow object


apple_parquet |>
 dplyr::mutate(
 a_check = stringr::str_detect(
 string = text,
 pattern = stringr::regex(
 pattern = "a",
 ignore_case = TRUE
 )
 ),
 p_count = stringr::str_count(
 string = text,
 pattern = stringr::regex(
 pattern = "p",
 ignore_case = TRUE
 )
 )
 ) |>
 dplyr::collect()
#> # A tibble: 2 × 3
#> text a_check p_count
#> <chr> <lgl> <int>
#> 1 apple TRUE 2
#> 2 APPLE FALSE 0

## Same result with stringr::fixed


apple_parquet |>
 dplyr::mutate(
 a_check = stringr::str_detect(
 string = text,
 pattern = stringr::fixed(
 pattern = "a",
 ignore_case = TRUE
 )
 ),
 p_count = stringr::str_count(
 string = text,
 pattern = stringr::fixed(
 pattern = "p",
 ignore_case = TRUE
 )
 )
 ) |>
 dplyr::collect()
#> # A tibble: 2 × 3
#> text a_check p_count
#> <chr> <lgl> <int>
#> 1 apple TRUE 2
#> 2 APPLE FALSE 0

## it works nicely just including the case insensitive in the regex

apple_parquet |>
 dplyr::mutate(
 a_check = stringr::str_detect(
 string = text,
 pattern = "(?i)a"
 ),
 p_count = stringr::str_count(
 string = text,
 pattern = "(?i)p"
 )
 ) |>
 dplyr::collect()
#> # A tibble: 2 × 3
#> text a_check p_count
#> <chr> <lgl> <int>
#> 1 apple TRUE 2
#> 2 APPLE TRUE 2



## With stringi

apple_df |>
 dplyr::mutate(
 a_check = stringi::stri_detect_regex(
 str = text,
 pattern = "a",
 case_insensitive = TRUE
 )
 ) |>
 dplyr::collect()
#> # A tibble: 2 × 2
#> text a_check
#> <chr> <lgl> 
#> 1 apple TRUE 
#> 2 APPLE TRUE



apple_parquet |>
 dplyr::mutate(
 a_check = stringi::stri_detect_regex(
 str = text,
 pattern = "a",
 case_insensitive = TRUE
 )
 ) |>
 dplyr::collect()
#> Error: Expression stringi::stri_detect_regex(str = text, pattern = "a", case_insensitive = TRUE) not supported in Arrow
#> Call collect() first to pull data into R.

^{Created on 2023-07-17 with reprex v2.0.2}

Component(s)

R

[thisisnic]

Thanks for reporting this @giocomai!

In this first example, I think it's how we implemented stringr::regex() - we need to update the code to work both with and without the stringr prefix:

library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#> filter, lag
#> The following objects are masked from 'package:base':
#> 
#> intersect, setdiff, setequal, union
library(arrow)
#> Some features are not enabled in this build of Arrow. Run `arrow_info()` for more information.
#> 
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#> 
#> timestamp

df <- tibble::tibble(x = "eXample")

# dplyr
df %>% 
 mutate(y = stringr::str_detect(x, pattern = stringr::regex("x", ignore_case = TRUE)))
#> # A tibble: 1 × 2
#> x y 
#> <chr> <lgl>
#> 1 eXample TRUE

# doesn't work
arrow_table(df) %>%
 mutate(y = stringr::str_detect(x, pattern = stringr::regex("x", ignore_case = TRUE))) %>%
 collect()
#> # A tibble: 1 × 2
#> x y 
#> <chr> <lgl>
#> 1 eXample FALSE

# works
arrow_table(df) %>%
 mutate(y = stringr::str_detect(x, pattern = regex("x", ignore_case = TRUE))) %>%
 collect()
#> # A tibble: 1 × 2
#> x y 
#> <chr> <lgl>
#> 1 eXample TRUE

^{Created on 2023-07-18 with reprex v2.0.2}

[thisisnic]

We have implemented fixed() similarly, so I'd expect similar results - works without prefix but not with - we should fix this.

[thisisnic]

We don't have bindings for stringi::str_detect_regex() in arrow (what we do support can be found here), which is the source of the error message for that function call.