ARROW-7412: [C++][Dataset] Provide FieldRef to disambiguate field references

[bkietz]

FieldRef is a new utility class which represents a reference to a field. It is intended to replace parameters like int field_index and const std::string& name; it can be implicitly constructed from either a field index or a name.

Nested fields can be referenced as well:

// the following all indicate schema->GetFieldByName("alpha")->type()->child(0)
FieldRef ref1({FieldRef("alpha"), FieldRef(0)});
FieldRef ref2("alpha", 0);
ARROW_ASSIGN_OR_RAISE(FieldRef ref3,
FieldRef::FromDotPath(".alpha[0]"));

FieldRefs provide a number of accessors for drilling down to potentially nested children. They are overloaded for convenience to support Schema (returns a field), DataType (returns a child field), Field (returns a child field of this field's type) Array (returns a child array), RecordBatch (returns a column), ChunkedArray (returns a ChunkedArray where each chunk is a child array of the corresponding original chunk) and Table (returns a column).

// Field names can match multiple fields in a Schema
Schema a_is_ambiguous({field("a", null()), field("a", null())});
auto matches = FieldRef("a").FindAll(a_is_ambiguous);
assert(matches.size() == 2);
assert_ok_and_eq(FieldRef::Get(match, a_is_ambiguous), a_is_ambiguous.field(0));
// Convenience accessor raises a helpful error if the field is not found or ambiguousARROW_ASSIGN_OR_RAISE(auto column, FieldRef("struct", "field_i32").GetOne(some_table));

[github-actions]

https://issues.apache.org/jira/browse/ARROW-7412

[pitrou]

I guess I'm mostly concerned about the variant injection into widely-used headers, and the lack of tests for small_vector.
I haven't looked into the FieldRef implementation closely.

[pitrou]

Hmm, it's a pity to start pulling the variant header in such a widely-used header...

[bkietz]

https://issues.apache.org/jira/browse/ARROW-8113

[nealrichardson]

@fsaintjacques you need this for your logical plan? Can you please review/merge?

[pitrou]

Thank you for the update. Two remaining questions below.

[pitrou]

Are we ignoring other potential matches here? FindAll could return multiple matches if there are columns with the same name, no?

[bkietz]

On line 103, I assert that there are not multiple matches

[pitrou]

Uh, looks like I forgot my glasses somewhere... where?

[bkietz]

CheckNonMultiple returns an error status if there are multiple matches

[fsaintjacques]

After offline conversation with Ben, Indices will be extracted in it's own class, representing an absolute path in a field.

[fsaintjacques]

The comment documentation is A+ for an important class like this.

[bkietz]

FieldRef is a new utility class which represents a reference to a field. It is intended to replace parameters like int field_index and const std::string& name; it can be implicitly constructed from either a field index or a name.

Nested fields can be referenced as well:

// the following all indicate schema->GetFieldByName("alpha")->type()->child(0)
FieldRef ref1({FieldRef("alpha"), FieldRef(0)});
FieldRef ref2("alpha", 0);
ARROW_ASSIGN_OR_RAISE(FieldRef ref3,
FieldRef::FromDotPath(".alpha[0]"));

FieldRefs provide a number of accessors for drilling down to potentially nested children. They are overloaded for convenience to support Schema (returns a field), DataType (returns a child field), Field (returns a child field of this field's type) Array (returns a child array), RecordBatch (returns a column), ChunkedArray (returns a ChunkedArray where each chunk is a child array of the corresponding original chunk) and Table (returns a column).

// Field names can match multiple fields in a Schema
Schema a_is_ambiguous({field("a", null()), field("a", null())});
auto matches = FieldRef("a").FindAll(a_is_ambiguous);
assert(matches.size() == 2);
assert_ok_and_eq(FieldRef::Get(match, a_is_ambiguous), a_is_ambiguous.field(0));
// Convenience accessor raises a helpful error if the field is not found or ambiguousARROW_ASSIGN_OR_RAISE(auto column, FieldRef("struct", "field_i32").GetOne(some_table));

[github-actions]

https://issues.apache.org/jira/browse/ARROW-7412

[pitrou]

I guess I'm mostly concerned about the variant injection into widely-used headers, and the lack of tests for small_vector.
I haven't looked into the FieldRef implementation closely.

[pitrou]

Hmm, it's a pity to start pulling the variant header in such a widely-used header...

[bkietz]

https://issues.apache.org/jira/browse/ARROW-8113

[nealrichardson]

@fsaintjacques you need this for your logical plan? Can you please review/merge?

[pitrou]

Thank you for the update. Two remaining questions below.

[pitrou]

Are we ignoring other potential matches here? FindAll could return multiple matches if there are columns with the same name, no?

[bkietz]

On line 103, I assert that there are not multiple matches

[pitrou]

Uh, looks like I forgot my glasses somewhere... where?

[bkietz]

CheckNonMultiple returns an error status if there are multiple matches

[fsaintjacques]

After offline conversation with Ben, Indices will be extracted in it's own class, representing an absolute path in a field.

[fsaintjacques]

The comment documentation is A+ for an important class like this.

[bkietz]

FieldRef is a new utility class which represents a reference to a field. It is intended to replace parameters like int field_index and const std::string& name; it can be implicitly constructed from either a field index or a name.

Nested fields can be referenced as well:

// the following all indicate schema->GetFieldByName("alpha")->type()->child(0)
FieldRef ref1({FieldRef("alpha"), FieldRef(0)});
FieldRef ref2("alpha", 0);
ARROW_ASSIGN_OR_RAISE(FieldRef ref3,
FieldRef::FromDotPath(".alpha[0]"));

FieldRefs provide a number of accessors for drilling down to potentially nested children. They are overloaded for convenience to support Schema (returns a field), DataType (returns a child field), Field (returns a child field of this field's type) Array (returns a child array), RecordBatch (returns a column), ChunkedArray (returns a ChunkedArray where each chunk is a child array of the corresponding original chunk) and Table (returns a column).

// Field names can match multiple fields in a Schema
Schema a_is_ambiguous({field("a", null()), field("a", null())});
auto matches = FieldRef("a").FindAll(a_is_ambiguous);
assert(matches.size() == 2);
assert_ok_and_eq(FieldRef::Get(match, a_is_ambiguous), a_is_ambiguous.field(0));
// Convenience accessor raises a helpful error if the field is not found or ambiguousARROW_ASSIGN_OR_RAISE(auto column, FieldRef("struct", "field_i32").GetOne(some_table));

[github-actions]

https://issues.apache.org/jira/browse/ARROW-7412

[pitrou]

I guess I'm mostly concerned about the variant injection into widely-used headers, and the lack of tests for small_vector.
I haven't looked into the FieldRef implementation closely.

[pitrou]

Hmm, it's a pity to start pulling the variant header in such a widely-used header...

[bkietz]

https://issues.apache.org/jira/browse/ARROW-8113

[nealrichardson]

@fsaintjacques you need this for your logical plan? Can you please review/merge?

[pitrou]

Thank you for the update. Two remaining questions below.

[pitrou]

Are we ignoring other potential matches here? FindAll could return multiple matches if there are columns with the same name, no?

[bkietz]

On line 103, I assert that there are not multiple matches

[pitrou]

Uh, looks like I forgot my glasses somewhere... where?

[bkietz]

CheckNonMultiple returns an error status if there are multiple matches

[fsaintjacques]

After offline conversation with Ben, Indices will be extracted in it's own class, representing an absolute path in a field.

[fsaintjacques]

The comment documentation is A+ for an important class like this.

[bkietz]

FieldRef is a new utility class which represents a reference to a field. It is intended to replace parameters like int field_index and const std::string& name; it can be implicitly constructed from either a field index or a name.

Nested fields can be referenced as well:

// the following all indicate schema->GetFieldByName("alpha")->type()->child(0)
FieldRef ref1({FieldRef("alpha"), FieldRef(0)});
FieldRef ref2("alpha", 0);
ARROW_ASSIGN_OR_RAISE(FieldRef ref3,
FieldRef::FromDotPath(".alpha[0]"));

FieldRefs provide a number of accessors for drilling down to potentially nested children. They are overloaded for convenience to support Schema (returns a field), DataType (returns a child field), Field (returns a child field of this field's type) Array (returns a child array), RecordBatch (returns a column), ChunkedArray (returns a ChunkedArray where each chunk is a child array of the corresponding original chunk) and Table (returns a column).

// Field names can match multiple fields in a Schema
Schema a_is_ambiguous({field("a", null()), field("a", null())});
auto matches = FieldRef("a").FindAll(a_is_ambiguous);
assert(matches.size() == 2);
assert_ok_and_eq(FieldRef::Get(match, a_is_ambiguous), a_is_ambiguous.field(0));
// Convenience accessor raises a helpful error if the field is not found or ambiguousARROW_ASSIGN_OR_RAISE(auto column, FieldRef("struct", "field_i32").GetOne(some_table));

[github-actions]

https://issues.apache.org/jira/browse/ARROW-7412

[pitrou]

I guess I'm mostly concerned about the variant injection into widely-used headers, and the lack of tests for small_vector.
I haven't looked into the FieldRef implementation closely.

[pitrou]

Hmm, it's a pity to start pulling the variant header in such a widely-used header...

[bkietz]

https://issues.apache.org/jira/browse/ARROW-8113

[nealrichardson]

@fsaintjacques you need this for your logical plan? Can you please review/merge?

[pitrou]

Thank you for the update. Two remaining questions below.

[pitrou]

Are we ignoring other potential matches here? FindAll could return multiple matches if there are columns with the same name, no?

[bkietz]

On line 103, I assert that there are not multiple matches

[pitrou]

Uh, looks like I forgot my glasses somewhere... where?

[bkietz]

CheckNonMultiple returns an error status if there are multiple matches

[fsaintjacques]

After offline conversation with Ben, Indices will be extracted in it's own class, representing an absolute path in a field.

[fsaintjacques]

The comment documentation is A+ for an important class like this.

[bkietz]

FieldRef is a new utility class which represents a reference to a field. It is intended to replace parameters like int field_index and const std::string& name; it can be implicitly constructed from either a field index or a name.

Nested fields can be referenced as well:

// the following all indicate schema->GetFieldByName("alpha")->type()->child(0)
FieldRef ref1({FieldRef("alpha"), FieldRef(0)});
FieldRef ref2("alpha", 0);
ARROW_ASSIGN_OR_RAISE(FieldRef ref3,
FieldRef::FromDotPath(".alpha[0]"));

FieldRefs provide a number of accessors for drilling down to potentially nested children. They are overloaded for convenience to support Schema (returns a field), DataType (returns a child field), Field (returns a child field of this field's type) Array (returns a child array), RecordBatch (returns a column), ChunkedArray (returns a ChunkedArray where each chunk is a child array of the corresponding original chunk) and Table (returns a column).

// Field names can match multiple fields in a Schema
Schema a_is_ambiguous({field("a", null()), field("a", null())});
auto matches = FieldRef("a").FindAll(a_is_ambiguous);
assert(matches.size() == 2);
assert_ok_and_eq(FieldRef::Get(match, a_is_ambiguous), a_is_ambiguous.field(0));
// Convenience accessor raises a helpful error if the field is not found or ambiguousARROW_ASSIGN_OR_RAISE(auto column, FieldRef("struct", "field_i32").GetOne(some_table));

[github-actions]

https://issues.apache.org/jira/browse/ARROW-7412

[pitrou]

I guess I'm mostly concerned about the variant injection into widely-used headers, and the lack of tests for small_vector.
I haven't looked into the FieldRef implementation closely.

[pitrou]

Hmm, it's a pity to start pulling the variant header in such a widely-used header...

[bkietz]

https://issues.apache.org/jira/browse/ARROW-8113

[nealrichardson]

@fsaintjacques you need this for your logical plan? Can you please review/merge?

[pitrou]

Thank you for the update. Two remaining questions below.

[pitrou]

Are we ignoring other potential matches here? FindAll could return multiple matches if there are columns with the same name, no?

[bkietz]

On line 103, I assert that there are not multiple matches

[pitrou]

Uh, looks like I forgot my glasses somewhere... where?

[bkietz]

CheckNonMultiple returns an error status if there are multiple matches

[fsaintjacques]

After offline conversation with Ben, Indices will be extracted in it's own class, representing an absolute path in a field.

[fsaintjacques]

The comment documentation is A+ for an important class like this.

[bkietz]

FieldRef is a new utility class which represents a reference to a field. It is intended to replace parameters like int field_index and const std::string& name; it can be implicitly constructed from either a field index or a name.

Nested fields can be referenced as well:

// the following all indicate schema->GetFieldByName("alpha")->type()->child(0)
FieldRef ref1({FieldRef("alpha"), FieldRef(0)});
FieldRef ref2("alpha", 0);
ARROW_ASSIGN_OR_RAISE(FieldRef ref3,
FieldRef::FromDotPath(".alpha[0]"));

FieldRefs provide a number of accessors for drilling down to potentially nested children. They are overloaded for convenience to support Schema (returns a field), DataType (returns a child field), Field (returns a child field of this field's type) Array (returns a child array), RecordBatch (returns a column), ChunkedArray (returns a ChunkedArray where each chunk is a child array of the corresponding original chunk) and Table (returns a column).

// Field names can match multiple fields in a Schema
Schema a_is_ambiguous({field("a", null()), field("a", null())});
auto matches = FieldRef("a").FindAll(a_is_ambiguous);
assert(matches.size() == 2);
assert_ok_and_eq(FieldRef::Get(match, a_is_ambiguous), a_is_ambiguous.field(0));
// Convenience accessor raises a helpful error if the field is not found or ambiguousARROW_ASSIGN_OR_RAISE(auto column, FieldRef("struct", "field_i32").GetOne(some_table));

[github-actions]

https://issues.apache.org/jira/browse/ARROW-7412

[pitrou]

I guess I'm mostly concerned about the variant injection into widely-used headers, and the lack of tests for small_vector.
I haven't looked into the FieldRef implementation closely.

[pitrou]

Hmm, it's a pity to start pulling the variant header in such a widely-used header...

[bkietz]

https://issues.apache.org/jira/browse/ARROW-8113

[nealrichardson]

@fsaintjacques you need this for your logical plan? Can you please review/merge?

[pitrou]

Thank you for the update. Two remaining questions below.

[pitrou]

Are we ignoring other potential matches here? FindAll could return multiple matches if there are columns with the same name, no?

[bkietz]

On line 103, I assert that there are not multiple matches

[pitrou]

Uh, looks like I forgot my glasses somewhere... where?

[bkietz]

CheckNonMultiple returns an error status if there are multiple matches

[fsaintjacques]

After offline conversation with Ben, Indices will be extracted in it's own class, representing an absolute path in a field.

[fsaintjacques]

The comment documentation is A+ for an important class like this.

[bkietz]

FieldRef is a new utility class which represents a reference to a field. It is intended to replace parameters like int field_index and const std::string& name; it can be implicitly constructed from either a field index or a name.

Nested fields can be referenced as well:

// the following all indicate schema->GetFieldByName("alpha")->type()->child(0)
FieldRef ref1({FieldRef("alpha"), FieldRef(0)});
FieldRef ref2("alpha", 0);
ARROW_ASSIGN_OR_RAISE(FieldRef ref3,
FieldRef::FromDotPath(".alpha[0]"));

FieldRefs provide a number of accessors for drilling down to potentially nested children. They are overloaded for convenience to support Schema (returns a field), DataType (returns a child field), Field (returns a child field of this field's type) Array (returns a child array), RecordBatch (returns a column), ChunkedArray (returns a ChunkedArray where each chunk is a child array of the corresponding original chunk) and Table (returns a column).

// Field names can match multiple fields in a Schema
Schema a_is_ambiguous({field("a", null()), field("a", null())});
auto matches = FieldRef("a").FindAll(a_is_ambiguous);
assert(matches.size() == 2);
assert_ok_and_eq(FieldRef::Get(match, a_is_ambiguous), a_is_ambiguous.field(0));
// Convenience accessor raises a helpful error if the field is not found or ambiguousARROW_ASSIGN_OR_RAISE(auto column, FieldRef("struct", "field_i32").GetOne(some_table));

[github-actions]

https://issues.apache.org/jira/browse/ARROW-7412

[pitrou]

I guess I'm mostly concerned about the variant injection into widely-used headers, and the lack of tests for small_vector.
I haven't looked into the FieldRef implementation closely.

[pitrou]

Hmm, it's a pity to start pulling the variant header in such a widely-used header...

[bkietz]

https://issues.apache.org/jira/browse/ARROW-8113

[nealrichardson]

@fsaintjacques you need this for your logical plan? Can you please review/merge?

[pitrou]

Thank you for the update. Two remaining questions below.

[pitrou]

Are we ignoring other potential matches here? FindAll could return multiple matches if there are columns with the same name, no?

[bkietz]

On line 103, I assert that there are not multiple matches

[pitrou]

Uh, looks like I forgot my glasses somewhere... where?

[bkietz]

CheckNonMultiple returns an error status if there are multiple matches

[fsaintjacques]

After offline conversation with Ben, Indices will be extracted in it's own class, representing an absolute path in a field.

[fsaintjacques]

The comment documentation is A+ for an important class like this.

[bkietz]

FieldRef is a new utility class which represents a reference to a field. It is intended to replace parameters like int field_index and const std::string& name; it can be implicitly constructed from either a field index or a name.

Nested fields can be referenced as well:

// the following all indicate schema->GetFieldByName("alpha")->type()->child(0)
FieldRef ref1({FieldRef("alpha"), FieldRef(0)});
FieldRef ref2("alpha", 0);
ARROW_ASSIGN_OR_RAISE(FieldRef ref3,
FieldRef::FromDotPath(".alpha[0]"));

FieldRefs provide a number of accessors for drilling down to potentially nested children. They are overloaded for convenience to support Schema (returns a field), DataType (returns a child field), Field (returns a child field of this field's type) Array (returns a child array), RecordBatch (returns a column), ChunkedArray (returns a ChunkedArray where each chunk is a child array of the corresponding original chunk) and Table (returns a column).

// Field names can match multiple fields in a Schema
Schema a_is_ambiguous({field("a", null()), field("a", null())});
auto matches = FieldRef("a").FindAll(a_is_ambiguous);
assert(matches.size() == 2);
assert_ok_and_eq(FieldRef::Get(match, a_is_ambiguous), a_is_ambiguous.field(0));
// Convenience accessor raises a helpful error if the field is not found or ambiguousARROW_ASSIGN_OR_RAISE(auto column, FieldRef("struct", "field_i32").GetOne(some_table));

[github-actions]

https://issues.apache.org/jira/browse/ARROW-7412

[pitrou]

I guess I'm mostly concerned about the variant injection into widely-used headers, and the lack of tests for small_vector.
I haven't looked into the FieldRef implementation closely.

[pitrou]

Hmm, it's a pity to start pulling the variant header in such a widely-used header...

[bkietz]

https://issues.apache.org/jira/browse/ARROW-8113

[nealrichardson]

@fsaintjacques you need this for your logical plan? Can you please review/merge?

[pitrou]

Thank you for the update. Two remaining questions below.

[pitrou]

Are we ignoring other potential matches here? FindAll could return multiple matches if there are columns with the same name, no?

[bkietz]

On line 103, I assert that there are not multiple matches

[pitrou]

Uh, looks like I forgot my glasses somewhere... where?

[bkietz]

CheckNonMultiple returns an error status if there are multiple matches

[fsaintjacques]

After offline conversation with Ben, Indices will be extracted in it's own class, representing an absolute path in a field.

[fsaintjacques]

The comment documentation is A+ for an important class like this.