Skip to content

ARROW-11259: [Python] Allow to create field reference to nested field - #12863

Closed
AlenkaF wants to merge 6 commits into
apache:masterfrom
AlenkaF:ARROW-11259
Closed

ARROW-11259: [Python] Allow to create field reference to nested field#12863
AlenkaF wants to merge 6 commits into
apache:masterfrom
AlenkaF:ARROW-11259

Conversation

@AlenkaF

Copy link
Copy Markdown
Member

@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@lidavidmlidavidm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you! I left a couple thoughts.

Comment threadpython/pyarrow/tests/test_dataset.py Outdated
Comment threadpython/pyarrow/compute.py Outdated
Comment threadpython/pyarrow/_compute.pyx Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have to spell out field_ref(FieldRef("foo", "bar")) explicitly or is it possible to simply write field_ref("foo", "bar")?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will try to simplify.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently it doesn't work, FieldRef("foo", "bar") needs to be spelled out.

Try 1: `field_ref("foo", "bar")`
```
/Users/alenkafrim/repos/arrow/cpp/src/arrow/compute/exec/expression_test.cc:1379:20: error: no matching function for call to 'field_ref'
ExpectRoundTrips(field_ref("foo", "bar", "baz"));
^~~~~~~~~
/Users/alenkafrim/repos/arrow/cpp/src/arrow/compute/exec/expression.h:152:12: note: candidate function not viable: requires single argument 'ref', but 3 arguments were provided
Expression field_ref(FieldRef ref);
```
Try 2: `field_ref(("foo", "bar"))`
```
/Users/alenkafrim/repos/arrow/cpp/src/arrow/compute/exec/expression_test.cc:1379:31: warning: expression result unused [-Wunused-value]
ExpectRoundTrips(field_ref(("foo", "bar", "baz")));
```

Comment threadpython/pyarrow/compute.py Outdated
@AlenkaF

Copy link
Copy Markdown
MemberAuthor

Thank you for the comments @lidavidm@pitrou!
I think this PR is ready for another round of review (the errors should be related to the S3fs) .

@pitroupitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update @AlenkaF ! Just two minor comments.

Comment threadpython/pyarrow/tests/test_dataset.py
Comment threadpython/pyarrow/tests/test_dataset.py Outdated
@AlenkaF
AlenkaF requested a review from pitrouApril 15, 2022 06:08

@pitroupitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, will merge if green. Thank you @AlenkaF !

@ursabot

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = 83dda5c and contender = e0668e2. e0668e2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.5% ⬆️0.04%] test-mac-arm
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.26% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/524|e0668e20 ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/511|e0668e20 test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/510|e0668e20 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/521|e0668e20 ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/523|83dda5c3 ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/510|83dda5c3 test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/509|83dda5c3 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/520|83dda5c3 ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@AlenkaF@ursabot@lidavidm@pitrou