Skip to content

ARROW-12198: [R] bindings for strptime#10334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
AlenkaF wants to merge 7 commits into from
Closed

ARROW-12198: [R] bindings for strptime #10334

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d04fbf2
ARROW-12198: [R] bindings for strptime
AlenkaF May 15, 2021
4aa1bc9
Moving strptime to nse_funcs and tests to dplyr_string_functions.
AlenkaF May 23, 2021
ff582ee
Adding check.tzone = FALSE
AlenkaF May 26, 2021
84765c3
Apply suggestions from code review
AlenkaF May 28, 2021
a271e6f
Apply suggestions from code review no. 2
AlenkaF Jun 1, 2021
14316f1
Merge branch 'ARROW-12198' of https://github.com/AlenkaF/arrow into A…
AlenkaF Jun 1, 2021
be6294f
tiny formatting change
jonkeane Jun 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions r/R/dplyr-functions.R
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -338,3 +338,20 @@ get_stringr_pattern_options <- function(pattern) {
contains_regex <- function(string) {
grepl("[.\\|()[{^$*+?]", string)
}

nse_funcs$strptime <- function(x, format = "%Y-%m-%d %H:%M:%S", tz = NULL, unit = "ms") {
# Arrow uses unit for time parsing, strptime() does not.
# Arrow has no default option for strptime (format, unit),
# we suggest following format = "%Y-%m-%d %H:%M:%S", unit = MILLI/1L/"ms",
# (ARROW-12809)

# ParseTimestampStrptime currently ignores the timezone information (ARROW-12820).
# Stop if tz is provided.
if (is.character(tz)) {
arrow_not_supported("Time zone argument")
}

unit <- make_valid_time_unit(unit, c(valid_time64_units, valid_time32_units))

Expression$create("strptime", x, options = list(format = format, unit = unit))
}
7 changes: 7 additions & 0 deletions r/src/compute.cpp
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -234,6 +234,13 @@ std::shared_ptr<arrow::compute::FunctionOptions> make_compute_options(
max_replacements);
}

if (func_name == "strptime") {
using Options = arrow::compute::StrptimeOptions;

@nealrichardsonnealrichardsonMay 16, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does StrptimeOptions have a Defaults() method in C++? If so, we should call it.

@AlenkaFAlenkaFMay 17, 2021

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that in scalar_string.cc line 1744 would suggest StrptimeOptions do not have Defaults() in C++?

@nealrichardsonnealrichardsonMay 17, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah looks like it. I made ARROW-12809 to evaluate whether that's correct, but for the purposes of the PR, it's fine.

return std::make_shared<Options>(
cpp11::as_cpp<std::string>(options["format"]),
cpp11::as_cpp<arrow::TimeUnit::type>(options["unit"]));
}

if (func_name == "split_pattern" || func_name == "split_pattern_regex") {
using Options = arrow::compute::SplitPatternOptions;
int64_t max_splits = -1;
Expand Down
75 changes: 75 additions & 0 deletions r/tests/testthat/test-dplyr-string-functions.R
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -493,3 +493,78 @@ test_that("edge cases in string detection and replacement", {
tibble(x = c("ABC"))
)
})

test_that("strptime", {

t_string <- tibble(x = c("2018-10-07 19:04:05", NA))
t_stamp <- tibble(x = c(lubridate::ymd_hms("2018-10-07 19:04:05"), NA))

expect_equal(
t_string %>%
Table$create() %>%
mutate(
x = strptime(x)
) %>%
collect(),
t_stamp,
check.tzone = FALSE
)

expect_equal(
t_string %>%
Table$create() %>%
mutate(
x = strptime(x, format = "%Y-%m-%d %H:%M:%S")
) %>%
collect(),
t_stamp,
check.tzone = FALSE
)

expect_equal(
t_string %>%
Table$create() %>%
mutate(
x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "ns")
) %>%
collect(),
t_stamp,
check.tzone = FALSE
)

expect_equal(
t_string %>%
Table$create() %>%
mutate(
x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "s")
) %>%
collect(),
t_stamp,
check.tzone = FALSE
)

tstring <- tibble(x = c("08-05-2008", NA))
tstamp <- tibble(x = c(strptime("08-05-2008", format = "%m-%d-%Y"), NA))

expect_equal(
tstring %>%
Table$create() %>%
mutate(
x = strptime(x, format = "%m-%d-%Y")
) %>%
collect(),
tstamp,
check.tzone = FALSE
)

})

test_that("errors in strptime", {
# Error when tz is passed

x <- Expression$field_ref("x")
expect_error(
nse_funcs$strptime(x, tz = "PDT"),
'Time zone argument not supported by Arrow'
)
})
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
ARROW-12198: [R] bindings for strptime by AlenkaF · Pull Request #10334 · apache/arrow · GitHub
Skip to content
ARROW-12198: [R] bindings for strptime#10334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

AlenkaF
wants to merge
7
commits into

from

 Closed


ARROW-12198: [R] bindings for strptime
#10334
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
d04fbf2

ARROW-12198: [R] bindings for strptime


AlenkaF May 15, 2021
4aa1bc9

Moving strptime to nse_funcs and tests to dplyr_string_functions.


AlenkaF May 23, 2021
ff582ee

Adding check.tzone = FALSE


AlenkaF May 26, 2021
84765c3

Apply suggestions from code review


AlenkaF May 28, 2021
a271e6f

Apply suggestions from code review no. 2


AlenkaF Jun 1, 2021
14316f1

Merge branch 'ARROW-12198' of https://github.com/AlenkaF/arrow into A…


AlenkaF Jun 1, 2021
be6294f

tiny formatting change


jonkeane Jun 4, 2021

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 17 additions & 0 deletions
r/R/dplyr-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -338,3 +338,20 @@ get_stringr_pattern_options <- function(pattern) {
contains_regex <- function(string) {
 grepl("[.\\|()[{^$*+?]", string)
}

nse_funcs$strptime <- function(x, format = "%Y-%m-%d %H:%M:%S", tz = NULL, unit = "ms") {
 # Arrow uses unit for time parsing, strptime() does not.
 # Arrow has no default option for strptime (format, unit),
 # we suggest following format = "%Y-%m-%d %H:%M:%S", unit = MILLI/1L/"ms",
 # (ARROW-12809)

 # ParseTimestampStrptime currently ignores the timezone information (ARROW-12820).
 # Stop if tz is provided.
 if (is.character(tz)) {
 arrow_not_supported("Time zone argument")
 }

 unit <- make_valid_time_unit(unit, c(valid_time64_units, valid_time32_units))

 Expression$create("strptime", x, options = list(format = format, unit = unit))
}

7 changes: 7 additions & 0 deletions
r/src/compute.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -234,6 +234,13 @@ std::shared_ptr<arrow::compute::FunctionOptions> make_compute_options(
 max_replacements);
 }

 if (func_name == "strptime") {
 using Options = arrow::compute::StrptimeOptions;
@nealrichardsonnealrichardsonMay 16, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does StrptimeOptions have a Defaults() method in C++? If so, we should call it.
@AlenkaFAlenkaFMay 17, 2021

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that in scalar_string.cc line 1744 would suggest StrptimeOptions do not have Defaults() in C++?
@nealrichardsonnealrichardsonMay 17, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah looks like it. I made ARROW-12809 to evaluate whether that's correct, but for the purposes of the PR, it's fine.
 return std::make_shared<Options>(
 cpp11::as_cpp<std::string>(options["format"]),
 cpp11::as_cpp<arrow::TimeUnit::type>(options["unit"]));
 }

 if (func_name == "split_pattern" || func_name == "split_pattern_regex") {
 using Options = arrow::compute::SplitPatternOptions;
 int64_t max_splits = -1;
Expand Down

75 changes: 75 additions & 0 deletions
r/tests/testthat/test-dplyr-string-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -493,3 +493,78 @@ test_that("edge cases in string detection and replacement", {
 tibble(x = c("ABC"))
 )
})

test_that("strptime", {

 t_string <- tibble(x = c("2018-10-07 19:04:05", NA))
 t_stamp <- tibble(x = c(lubridate::ymd_hms("2018-10-07 19:04:05"), NA))

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x)
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "ns")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "s")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 tstring <- tibble(x = c("08-05-2008", NA))
 tstamp <- tibble(x = c(strptime("08-05-2008", format = "%m-%d-%Y"), NA))

 expect_equal(
 tstring %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%m-%d-%Y")
 ) %>%
 collect(),
 tstamp,
 check.tzone = FALSE
 )

})

test_that("errors in strptime", {
 # Error when tz is passed

 x <- Expression$field_ref("x")
 expect_error(
 nse_funcs$strptime(x, tz = "PDT"),
 'Time zone argument not supported by Arrow'
 )
})

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ARROW-12198: [R] bindings for strptime by AlenkaF · Pull Request #10334 · apache/arrow · GitHub
Skip to content
ARROW-12198: [R] bindings for strptime#10334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

AlenkaF
wants to merge
7
commits into

from

 Closed


ARROW-12198: [R] bindings for strptime
#10334
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
d04fbf2

ARROW-12198: [R] bindings for strptime


AlenkaF May 15, 2021
4aa1bc9

Moving strptime to nse_funcs and tests to dplyr_string_functions.


AlenkaF May 23, 2021
ff582ee

Adding check.tzone = FALSE


AlenkaF May 26, 2021
84765c3

Apply suggestions from code review


AlenkaF May 28, 2021
a271e6f

Apply suggestions from code review no. 2


AlenkaF Jun 1, 2021
14316f1

Merge branch 'ARROW-12198' of https://github.com/AlenkaF/arrow into A…


AlenkaF Jun 1, 2021
be6294f

tiny formatting change


jonkeane Jun 4, 2021

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 17 additions & 0 deletions
r/R/dplyr-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -338,3 +338,20 @@ get_stringr_pattern_options <- function(pattern) {
contains_regex <- function(string) {
 grepl("[.\\|()[{^$*+?]", string)
}

nse_funcs$strptime <- function(x, format = "%Y-%m-%d %H:%M:%S", tz = NULL, unit = "ms") {
 # Arrow uses unit for time parsing, strptime() does not.
 # Arrow has no default option for strptime (format, unit),
 # we suggest following format = "%Y-%m-%d %H:%M:%S", unit = MILLI/1L/"ms",
 # (ARROW-12809)

 # ParseTimestampStrptime currently ignores the timezone information (ARROW-12820).
 # Stop if tz is provided.
 if (is.character(tz)) {
 arrow_not_supported("Time zone argument")
 }

 unit <- make_valid_time_unit(unit, c(valid_time64_units, valid_time32_units))

 Expression$create("strptime", x, options = list(format = format, unit = unit))
}

7 changes: 7 additions & 0 deletions
r/src/compute.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -234,6 +234,13 @@ std::shared_ptr<arrow::compute::FunctionOptions> make_compute_options(
 max_replacements);
 }

 if (func_name == "strptime") {
 using Options = arrow::compute::StrptimeOptions;
@nealrichardsonnealrichardsonMay 16, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does StrptimeOptions have a Defaults() method in C++? If so, we should call it.
@AlenkaFAlenkaFMay 17, 2021

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that in scalar_string.cc line 1744 would suggest StrptimeOptions do not have Defaults() in C++?
@nealrichardsonnealrichardsonMay 17, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah looks like it. I made ARROW-12809 to evaluate whether that's correct, but for the purposes of the PR, it's fine.
 return std::make_shared<Options>(
 cpp11::as_cpp<std::string>(options["format"]),
 cpp11::as_cpp<arrow::TimeUnit::type>(options["unit"]));
 }

 if (func_name == "split_pattern" || func_name == "split_pattern_regex") {
 using Options = arrow::compute::SplitPatternOptions;
 int64_t max_splits = -1;
Expand Down

75 changes: 75 additions & 0 deletions
r/tests/testthat/test-dplyr-string-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -493,3 +493,78 @@ test_that("edge cases in string detection and replacement", {
 tibble(x = c("ABC"))
 )
})

test_that("strptime", {

 t_string <- tibble(x = c("2018-10-07 19:04:05", NA))
 t_stamp <- tibble(x = c(lubridate::ymd_hms("2018-10-07 19:04:05"), NA))

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x)
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "ns")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "s")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 tstring <- tibble(x = c("08-05-2008", NA))
 tstamp <- tibble(x = c(strptime("08-05-2008", format = "%m-%d-%Y"), NA))

 expect_equal(
 tstring %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%m-%d-%Y")
 ) %>%
 collect(),
 tstamp,
 check.tzone = FALSE
 )

})

test_that("errors in strptime", {
 # Error when tz is passed

 x <- Expression$field_ref("x")
 expect_error(
 nse_funcs$strptime(x, tz = "PDT"),
 'Time zone argument not supported by Arrow'
 )
})

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ARROW-12198: [R] bindings for strptime by AlenkaF · Pull Request #10334 · apache/arrow · GitHub
Skip to content
ARROW-12198: [R] bindings for strptime#10334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

AlenkaF
wants to merge
7
commits into

from

 Closed


ARROW-12198: [R] bindings for strptime
#10334
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
d04fbf2

ARROW-12198: [R] bindings for strptime


AlenkaF May 15, 2021
4aa1bc9

Moving strptime to nse_funcs and tests to dplyr_string_functions.


AlenkaF May 23, 2021
ff582ee

Adding check.tzone = FALSE


AlenkaF May 26, 2021
84765c3

Apply suggestions from code review


AlenkaF May 28, 2021
a271e6f

Apply suggestions from code review no. 2


AlenkaF Jun 1, 2021
14316f1

Merge branch 'ARROW-12198' of https://github.com/AlenkaF/arrow into A…


AlenkaF Jun 1, 2021
be6294f

tiny formatting change


jonkeane Jun 4, 2021

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 17 additions & 0 deletions
r/R/dplyr-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -338,3 +338,20 @@ get_stringr_pattern_options <- function(pattern) {
contains_regex <- function(string) {
 grepl("[.\\|()[{^$*+?]", string)
}

nse_funcs$strptime <- function(x, format = "%Y-%m-%d %H:%M:%S", tz = NULL, unit = "ms") {
 # Arrow uses unit for time parsing, strptime() does not.
 # Arrow has no default option for strptime (format, unit),
 # we suggest following format = "%Y-%m-%d %H:%M:%S", unit = MILLI/1L/"ms",
 # (ARROW-12809)

 # ParseTimestampStrptime currently ignores the timezone information (ARROW-12820).
 # Stop if tz is provided.
 if (is.character(tz)) {
 arrow_not_supported("Time zone argument")
 }

 unit <- make_valid_time_unit(unit, c(valid_time64_units, valid_time32_units))

 Expression$create("strptime", x, options = list(format = format, unit = unit))
}

7 changes: 7 additions & 0 deletions
r/src/compute.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -234,6 +234,13 @@ std::shared_ptr<arrow::compute::FunctionOptions> make_compute_options(
 max_replacements);
 }

 if (func_name == "strptime") {
 using Options = arrow::compute::StrptimeOptions;
@nealrichardsonnealrichardsonMay 16, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does StrptimeOptions have a Defaults() method in C++? If so, we should call it.
@AlenkaFAlenkaFMay 17, 2021

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that in scalar_string.cc line 1744 would suggest StrptimeOptions do not have Defaults() in C++?
@nealrichardsonnealrichardsonMay 17, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah looks like it. I made ARROW-12809 to evaluate whether that's correct, but for the purposes of the PR, it's fine.
 return std::make_shared<Options>(
 cpp11::as_cpp<std::string>(options["format"]),
 cpp11::as_cpp<arrow::TimeUnit::type>(options["unit"]));
 }

 if (func_name == "split_pattern" || func_name == "split_pattern_regex") {
 using Options = arrow::compute::SplitPatternOptions;
 int64_t max_splits = -1;
Expand Down

75 changes: 75 additions & 0 deletions
r/tests/testthat/test-dplyr-string-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -493,3 +493,78 @@ test_that("edge cases in string detection and replacement", {
 tibble(x = c("ABC"))
 )
})

test_that("strptime", {

 t_string <- tibble(x = c("2018-10-07 19:04:05", NA))
 t_stamp <- tibble(x = c(lubridate::ymd_hms("2018-10-07 19:04:05"), NA))

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x)
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "ns")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "s")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 tstring <- tibble(x = c("08-05-2008", NA))
 tstamp <- tibble(x = c(strptime("08-05-2008", format = "%m-%d-%Y"), NA))

 expect_equal(
 tstring %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%m-%d-%Y")
 ) %>%
 collect(),
 tstamp,
 check.tzone = FALSE
 )

})

test_that("errors in strptime", {
 # Error when tz is passed

 x <- Expression$field_ref("x")
 expect_error(
 nse_funcs$strptime(x, tz = "PDT"),
 'Time zone argument not supported by Arrow'
 )
})

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
ARROW-12198: [R] bindings for strptime by AlenkaF · Pull Request #10334 · apache/arrow · GitHub
Skip to content
ARROW-12198: [R] bindings for strptime#10334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

AlenkaF
wants to merge
7
commits into

from

 Closed


ARROW-12198: [R] bindings for strptime
#10334
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
d04fbf2

ARROW-12198: [R] bindings for strptime


AlenkaF May 15, 2021
4aa1bc9

Moving strptime to nse_funcs and tests to dplyr_string_functions.


AlenkaF May 23, 2021
ff582ee

Adding check.tzone = FALSE


AlenkaF May 26, 2021
84765c3

Apply suggestions from code review


AlenkaF May 28, 2021
a271e6f

Apply suggestions from code review no. 2


AlenkaF Jun 1, 2021
14316f1

Merge branch 'ARROW-12198' of https://github.com/AlenkaF/arrow into A…


AlenkaF Jun 1, 2021
be6294f

tiny formatting change


jonkeane Jun 4, 2021

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 17 additions & 0 deletions
r/R/dplyr-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -338,3 +338,20 @@ get_stringr_pattern_options <- function(pattern) {
contains_regex <- function(string) {
 grepl("[.\\|()[{^$*+?]", string)
}

nse_funcs$strptime <- function(x, format = "%Y-%m-%d %H:%M:%S", tz = NULL, unit = "ms") {
 # Arrow uses unit for time parsing, strptime() does not.
 # Arrow has no default option for strptime (format, unit),
 # we suggest following format = "%Y-%m-%d %H:%M:%S", unit = MILLI/1L/"ms",
 # (ARROW-12809)

 # ParseTimestampStrptime currently ignores the timezone information (ARROW-12820).
 # Stop if tz is provided.
 if (is.character(tz)) {
 arrow_not_supported("Time zone argument")
 }

 unit <- make_valid_time_unit(unit, c(valid_time64_units, valid_time32_units))

 Expression$create("strptime", x, options = list(format = format, unit = unit))
}

7 changes: 7 additions & 0 deletions
r/src/compute.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -234,6 +234,13 @@ std::shared_ptr<arrow::compute::FunctionOptions> make_compute_options(
 max_replacements);
 }

 if (func_name == "strptime") {
 using Options = arrow::compute::StrptimeOptions;
@nealrichardsonnealrichardsonMay 16, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does StrptimeOptions have a Defaults() method in C++? If so, we should call it.
@AlenkaFAlenkaFMay 17, 2021

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that in scalar_string.cc line 1744 would suggest StrptimeOptions do not have Defaults() in C++?
@nealrichardsonnealrichardsonMay 17, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah looks like it. I made ARROW-12809 to evaluate whether that's correct, but for the purposes of the PR, it's fine.
 return std::make_shared<Options>(
 cpp11::as_cpp<std::string>(options["format"]),
 cpp11::as_cpp<arrow::TimeUnit::type>(options["unit"]));
 }

 if (func_name == "split_pattern" || func_name == "split_pattern_regex") {
 using Options = arrow::compute::SplitPatternOptions;
 int64_t max_splits = -1;
Expand Down

75 changes: 75 additions & 0 deletions
r/tests/testthat/test-dplyr-string-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -493,3 +493,78 @@ test_that("edge cases in string detection and replacement", {
 tibble(x = c("ABC"))
 )
})

test_that("strptime", {

 t_string <- tibble(x = c("2018-10-07 19:04:05", NA))
 t_stamp <- tibble(x = c(lubridate::ymd_hms("2018-10-07 19:04:05"), NA))

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x)
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "ns")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "s")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 tstring <- tibble(x = c("08-05-2008", NA))
 tstamp <- tibble(x = c(strptime("08-05-2008", format = "%m-%d-%Y"), NA))

 expect_equal(
 tstring %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%m-%d-%Y")
 ) %>%
 collect(),
 tstamp,
 check.tzone = FALSE
 )

})

test_that("errors in strptime", {
 # Error when tz is passed

 x <- Expression$field_ref("x")
 expect_error(
 nse_funcs$strptime(x, tz = "PDT"),
 'Time zone argument not supported by Arrow'
 )
})

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ARROW-12198: [R] bindings for strptime by AlenkaF · Pull Request #10334 · apache/arrow · GitHub
Skip to content
ARROW-12198: [R] bindings for strptime#10334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

AlenkaF
wants to merge
7
commits into

from

 Closed


ARROW-12198: [R] bindings for strptime
#10334
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
d04fbf2

ARROW-12198: [R] bindings for strptime


AlenkaF May 15, 2021
4aa1bc9

Moving strptime to nse_funcs and tests to dplyr_string_functions.


AlenkaF May 23, 2021
ff582ee

Adding check.tzone = FALSE


AlenkaF May 26, 2021
84765c3

Apply suggestions from code review


AlenkaF May 28, 2021
a271e6f

Apply suggestions from code review no. 2


AlenkaF Jun 1, 2021
14316f1

Merge branch 'ARROW-12198' of https://github.com/AlenkaF/arrow into A…


AlenkaF Jun 1, 2021
be6294f

tiny formatting change


jonkeane Jun 4, 2021

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 17 additions & 0 deletions
r/R/dplyr-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -338,3 +338,20 @@ get_stringr_pattern_options <- function(pattern) {
contains_regex <- function(string) {
 grepl("[.\\|()[{^$*+?]", string)
}

nse_funcs$strptime <- function(x, format = "%Y-%m-%d %H:%M:%S", tz = NULL, unit = "ms") {
 # Arrow uses unit for time parsing, strptime() does not.
 # Arrow has no default option for strptime (format, unit),
 # we suggest following format = "%Y-%m-%d %H:%M:%S", unit = MILLI/1L/"ms",
 # (ARROW-12809)

 # ParseTimestampStrptime currently ignores the timezone information (ARROW-12820).
 # Stop if tz is provided.
 if (is.character(tz)) {
 arrow_not_supported("Time zone argument")
 }

 unit <- make_valid_time_unit(unit, c(valid_time64_units, valid_time32_units))

 Expression$create("strptime", x, options = list(format = format, unit = unit))
}

7 changes: 7 additions & 0 deletions
r/src/compute.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -234,6 +234,13 @@ std::shared_ptr<arrow::compute::FunctionOptions> make_compute_options(
 max_replacements);
 }

 if (func_name == "strptime") {
 using Options = arrow::compute::StrptimeOptions;
@nealrichardsonnealrichardsonMay 16, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does StrptimeOptions have a Defaults() method in C++? If so, we should call it.
@AlenkaFAlenkaFMay 17, 2021

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that in scalar_string.cc line 1744 would suggest StrptimeOptions do not have Defaults() in C++?
@nealrichardsonnealrichardsonMay 17, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah looks like it. I made ARROW-12809 to evaluate whether that's correct, but for the purposes of the PR, it's fine.
 return std::make_shared<Options>(
 cpp11::as_cpp<std::string>(options["format"]),
 cpp11::as_cpp<arrow::TimeUnit::type>(options["unit"]));
 }

 if (func_name == "split_pattern" || func_name == "split_pattern_regex") {
 using Options = arrow::compute::SplitPatternOptions;
 int64_t max_splits = -1;
Expand Down

75 changes: 75 additions & 0 deletions
r/tests/testthat/test-dplyr-string-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -493,3 +493,78 @@ test_that("edge cases in string detection and replacement", {
 tibble(x = c("ABC"))
 )
})

test_that("strptime", {

 t_string <- tibble(x = c("2018-10-07 19:04:05", NA))
 t_stamp <- tibble(x = c(lubridate::ymd_hms("2018-10-07 19:04:05"), NA))

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x)
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "ns")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "s")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 tstring <- tibble(x = c("08-05-2008", NA))
 tstamp <- tibble(x = c(strptime("08-05-2008", format = "%m-%d-%Y"), NA))

 expect_equal(
 tstring %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%m-%d-%Y")
 ) %>%
 collect(),
 tstamp,
 check.tzone = FALSE
 )

})

test_that("errors in strptime", {
 # Error when tz is passed

 x <- Expression$field_ref("x")
 expect_error(
 nse_funcs$strptime(x, tz = "PDT"),
 'Time zone argument not supported by Arrow'
 )
})

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
ARROW-12198: [R] bindings for strptime by AlenkaF · Pull Request #10334 · apache/arrow · GitHub
Skip to content
ARROW-12198: [R] bindings for strptime#10334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

AlenkaF
wants to merge
7
commits into

from

 Closed


ARROW-12198: [R] bindings for strptime
#10334
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
d04fbf2

ARROW-12198: [R] bindings for strptime


AlenkaF May 15, 2021
4aa1bc9

Moving strptime to nse_funcs and tests to dplyr_string_functions.


AlenkaF May 23, 2021
ff582ee

Adding check.tzone = FALSE


AlenkaF May 26, 2021
84765c3

Apply suggestions from code review


AlenkaF May 28, 2021
a271e6f

Apply suggestions from code review no. 2


AlenkaF Jun 1, 2021
14316f1

Merge branch 'ARROW-12198' of https://github.com/AlenkaF/arrow into A…


AlenkaF Jun 1, 2021
be6294f

tiny formatting change


jonkeane Jun 4, 2021

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 17 additions & 0 deletions
r/R/dplyr-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -338,3 +338,20 @@ get_stringr_pattern_options <- function(pattern) {
contains_regex <- function(string) {
 grepl("[.\\|()[{^$*+?]", string)
}

nse_funcs$strptime <- function(x, format = "%Y-%m-%d %H:%M:%S", tz = NULL, unit = "ms") {
 # Arrow uses unit for time parsing, strptime() does not.
 # Arrow has no default option for strptime (format, unit),
 # we suggest following format = "%Y-%m-%d %H:%M:%S", unit = MILLI/1L/"ms",
 # (ARROW-12809)

 # ParseTimestampStrptime currently ignores the timezone information (ARROW-12820).
 # Stop if tz is provided.
 if (is.character(tz)) {
 arrow_not_supported("Time zone argument")
 }

 unit <- make_valid_time_unit(unit, c(valid_time64_units, valid_time32_units))

 Expression$create("strptime", x, options = list(format = format, unit = unit))
}

7 changes: 7 additions & 0 deletions
r/src/compute.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -234,6 +234,13 @@ std::shared_ptr<arrow::compute::FunctionOptions> make_compute_options(
 max_replacements);
 }

 if (func_name == "strptime") {
 using Options = arrow::compute::StrptimeOptions;
@nealrichardsonnealrichardsonMay 16, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does StrptimeOptions have a Defaults() method in C++? If so, we should call it.
@AlenkaFAlenkaFMay 17, 2021

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that in scalar_string.cc line 1744 would suggest StrptimeOptions do not have Defaults() in C++?
@nealrichardsonnealrichardsonMay 17, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah looks like it. I made ARROW-12809 to evaluate whether that's correct, but for the purposes of the PR, it's fine.
 return std::make_shared<Options>(
 cpp11::as_cpp<std::string>(options["format"]),
 cpp11::as_cpp<arrow::TimeUnit::type>(options["unit"]));
 }

 if (func_name == "split_pattern" || func_name == "split_pattern_regex") {
 using Options = arrow::compute::SplitPatternOptions;
 int64_t max_splits = -1;
Expand Down

75 changes: 75 additions & 0 deletions
r/tests/testthat/test-dplyr-string-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -493,3 +493,78 @@ test_that("edge cases in string detection and replacement", {
 tibble(x = c("ABC"))
 )
})

test_that("strptime", {

 t_string <- tibble(x = c("2018-10-07 19:04:05", NA))
 t_stamp <- tibble(x = c(lubridate::ymd_hms("2018-10-07 19:04:05"), NA))

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x)
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "ns")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "s")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 tstring <- tibble(x = c("08-05-2008", NA))
 tstamp <- tibble(x = c(strptime("08-05-2008", format = "%m-%d-%Y"), NA))

 expect_equal(
 tstring %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%m-%d-%Y")
 ) %>%
 collect(),
 tstamp,
 check.tzone = FALSE
 )

})

test_that("errors in strptime", {
 # Error when tz is passed

 x <- Expression$field_ref("x")
 expect_error(
 nse_funcs$strptime(x, tz = "PDT"),
 'Time zone argument not supported by Arrow'
 )
})

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
ARROW-12198: [R] bindings for strptime by AlenkaF · Pull Request #10334 · apache/arrow · GitHub
Skip to content
ARROW-12198: [R] bindings for strptime#10334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Closed

AlenkaF
wants to merge
7
commits into

from

 Closed


ARROW-12198: [R] bindings for strptime
#10334
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


7 commits


Select commit
Hold shift + click to select a range
d04fbf2

ARROW-12198: [R] bindings for strptime


AlenkaF May 15, 2021
4aa1bc9

Moving strptime to nse_funcs and tests to dplyr_string_functions.


AlenkaF May 23, 2021
ff582ee

Adding check.tzone = FALSE


AlenkaF May 26, 2021
84765c3

Apply suggestions from code review


AlenkaF May 28, 2021
a271e6f

Apply suggestions from code review no. 2


AlenkaF Jun 1, 2021
14316f1

Merge branch 'ARROW-12198' of https://github.com/AlenkaF/arrow into A…


AlenkaF Jun 1, 2021
be6294f

tiny formatting change


jonkeane Jun 4, 2021

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


17 changes: 17 additions & 0 deletions
r/R/dplyr-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -338,3 +338,20 @@ get_stringr_pattern_options <- function(pattern) {
contains_regex <- function(string) {
 grepl("[.\\|()[{^$*+?]", string)
}

nse_funcs$strptime <- function(x, format = "%Y-%m-%d %H:%M:%S", tz = NULL, unit = "ms") {
 # Arrow uses unit for time parsing, strptime() does not.
 # Arrow has no default option for strptime (format, unit),
 # we suggest following format = "%Y-%m-%d %H:%M:%S", unit = MILLI/1L/"ms",
 # (ARROW-12809)

 # ParseTimestampStrptime currently ignores the timezone information (ARROW-12820).
 # Stop if tz is provided.
 if (is.character(tz)) {
 arrow_not_supported("Time zone argument")
 }

 unit <- make_valid_time_unit(unit, c(valid_time64_units, valid_time32_units))

 Expression$create("strptime", x, options = list(format = format, unit = unit))
}

7 changes: 7 additions & 0 deletions
r/src/compute.cpp

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -234,6 +234,13 @@ std::shared_ptr<arrow::compute::FunctionOptions> make_compute_options(
 max_replacements);
 }

 if (func_name == "strptime") {
 using Options = arrow::compute::StrptimeOptions;
@nealrichardsonnealrichardsonMay 16, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does StrptimeOptions have a Defaults() method in C++? If so, we should call it.
@AlenkaFAlenkaFMay 17, 2021

Copy link


Copy Markdown

MemberAuthor
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that in scalar_string.cc line 1744 would suggest StrptimeOptions do not have Defaults() in C++?
@nealrichardsonnealrichardsonMay 17, 2021

Copy link


Copy Markdown

Member
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah looks like it. I made ARROW-12809 to evaluate whether that's correct, but for the purposes of the PR, it's fine.
 return std::make_shared<Options>(
 cpp11::as_cpp<std::string>(options["format"]),
 cpp11::as_cpp<arrow::TimeUnit::type>(options["unit"]));
 }

 if (func_name == "split_pattern" || func_name == "split_pattern_regex") {
 using Options = arrow::compute::SplitPatternOptions;
 int64_t max_splits = -1;
Expand Down

75 changes: 75 additions & 0 deletions
r/tests/testthat/test-dplyr-string-functions.R

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -493,3 +493,78 @@ test_that("edge cases in string detection and replacement", {
 tibble(x = c("ABC"))
 )
})

test_that("strptime", {

 t_string <- tibble(x = c("2018-10-07 19:04:05", NA))
 t_stamp <- tibble(x = c(lubridate::ymd_hms("2018-10-07 19:04:05"), NA))

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x)
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "ns")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 expect_equal(
 t_string %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%Y-%m-%d %H:%M:%S", unit = "s")
 ) %>%
 collect(),
 t_stamp,
 check.tzone = FALSE
 )

 tstring <- tibble(x = c("08-05-2008", NA))
 tstamp <- tibble(x = c(strptime("08-05-2008", format = "%m-%d-%Y"), NA))

 expect_equal(
 tstring %>%
 Table$create() %>%
 mutate(
 x = strptime(x, format = "%m-%d-%Y")
 ) %>%
 collect(),
 tstamp,
 check.tzone = FALSE
 )

})

test_that("errors in strptime", {
 # Error when tz is passed

 x <- Expression$field_ref("x")
 expect_error(
 nse_funcs$strptime(x, tz = "PDT"),
 'Time zone argument not supported by Arrow'
 )
})