GH-44555: [C++][Compute] Allow casting struct to bigger nullable struct by Tom-Newton · Pull Request #44587 · apache/arrow

Tom-Newton · 2024-10-30T18:14:55Z

Rationale for this change

Sometimes its useful to add a column full of nulls in a cast. Currently this is supported in top level columns but not inside structs. Example where this is important: delta-io/delta-rs#1610

What changes are included in this PR?

Add support for filling in columns full of null for nullable struct fields.
I've gone for a fairly minimal change that achieves what I needed but I wonder if there should be a more significant change so that this casting is done by field name and ignore the field order.

Are these changes tested?

Yes. The expected behaviour in several existing tests has been altered and a couple of new tests have been added.

I also rolled out a custom build with this change internally because it suddenly became a critical problem.

Are there any user-facing changes?

Yes. There are scenarios that previously failed with struct fields don't match or are in the wrong order but now succeed after filling in nulls.

GitHub Issue: [C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555

github-actions · 2024-10-30T18:15:23Z

⚠️ GitHub issue #44555has been automatically assigned in GitHub to PR creator.

Tom-Newton · 2024-10-30T18:50:17Z

I'm not confident that I've taken the right approach here but I think its ready for review.

Tom-Newton · 2024-10-30T19:49:40Z

Ok, it looks like there is at least one more tests I need to update. I can probably sort that tomorrow.

Tom-Newton · 2024-11-01T10:55:27Z

 field("c", struct_({
 field("d", int64()),
- field("e", float64()),
+ field("e", int64()),


I suspect this was not supposed to be different from in the physical schema. This data is used to test if a virtual column will be materialised so I guess it shouldn't be testing int64 -> float64 casting at the same time.

Tom-Newton · 2024-11-11T16:32:58Z

Sorry for the direct ping but @lidavidm please could you review when you get a chance. I assume you are the right person to review this?

lidavidm

Looks reasonable to me, thanks. CC @bkietz for some more eyes?

Co-authored-by: David Li <li.davidm96@gmail.com>

lidavidm · 2024-11-25T00:59:02Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

conbench-apache-arrow · 2024-11-25T07:06:42Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c14d55d.

There were 132 benchmark results with an error:

Commit Run on arm64-t4g-2xlarge-linux at 2024-11-25 02:34:37Z
- tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
- tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1
and 130 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

Tom-Newton · 2024-11-25T07:20:34Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

Thanks for reviewing 🙂

Tom-Newton · 2024-10-30T18:14:55Z

Rationale for this change

Sometimes its useful to add a column full of nulls in a cast. Currently this is supported in top level columns but not inside structs. Example where this is important: delta-io/delta-rs#1610

What changes are included in this PR?

Add support for filling in columns full of null for nullable struct fields.
I've gone for a fairly minimal change that achieves what I needed but I wonder if there should be a more significant change so that this casting is done by field name and ignore the field order.

Are these changes tested?

Yes. The expected behaviour in several existing tests has been altered and a couple of new tests have been added.

I also rolled out a custom build with this change internally because it suddenly became a critical problem.

Are there any user-facing changes?

Yes. There are scenarios that previously failed with struct fields don't match or are in the wrong order but now succeed after filling in nulls.

GitHub Issue: [C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555

github-actions · 2024-10-30T18:15:23Z

⚠️ GitHub issue #44555has been automatically assigned in GitHub to PR creator.

Tom-Newton · 2024-10-30T18:50:17Z

I'm not confident that I've taken the right approach here but I think its ready for review.

Tom-Newton · 2024-10-30T19:49:40Z

Ok, it looks like there is at least one more tests I need to update. I can probably sort that tomorrow.

Tom-Newton · 2024-11-01T10:55:27Z

 field("c", struct_({
 field("d", int64()),
- field("e", float64()),
+ field("e", int64()),


I suspect this was not supposed to be different from in the physical schema. This data is used to test if a virtual column will be materialised so I guess it shouldn't be testing int64 -> float64 casting at the same time.

Tom-Newton · 2024-11-11T16:32:58Z

Sorry for the direct ping but @lidavidm please could you review when you get a chance. I assume you are the right person to review this?

lidavidm

Looks reasonable to me, thanks. CC @bkietz for some more eyes?

Co-authored-by: David Li <li.davidm96@gmail.com>

lidavidm · 2024-11-25T00:59:02Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

conbench-apache-arrow · 2024-11-25T07:06:42Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c14d55d.

There were 132 benchmark results with an error:

Commit Run on arm64-t4g-2xlarge-linux at 2024-11-25 02:34:37Z
- tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
- tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1
and 130 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

Tom-Newton · 2024-11-25T07:20:34Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

Thanks for reviewing 🙂

Tom-Newton · 2024-10-30T18:14:55Z

Rationale for this change

Sometimes its useful to add a column full of nulls in a cast. Currently this is supported in top level columns but not inside structs. Example where this is important: delta-io/delta-rs#1610

What changes are included in this PR?

Add support for filling in columns full of null for nullable struct fields.
I've gone for a fairly minimal change that achieves what I needed but I wonder if there should be a more significant change so that this casting is done by field name and ignore the field order.

Are these changes tested?

Yes. The expected behaviour in several existing tests has been altered and a couple of new tests have been added.

I also rolled out a custom build with this change internally because it suddenly became a critical problem.

Are there any user-facing changes?

Yes. There are scenarios that previously failed with struct fields don't match or are in the wrong order but now succeed after filling in nulls.

GitHub Issue: [C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555

github-actions · 2024-10-30T18:15:23Z

⚠️ GitHub issue #44555has been automatically assigned in GitHub to PR creator.

Tom-Newton · 2024-10-30T18:50:17Z

I'm not confident that I've taken the right approach here but I think its ready for review.

Tom-Newton · 2024-10-30T19:49:40Z

Ok, it looks like there is at least one more tests I need to update. I can probably sort that tomorrow.

Tom-Newton · 2024-11-01T10:55:27Z

 field("c", struct_({
 field("d", int64()),
- field("e", float64()),
+ field("e", int64()),


I suspect this was not supposed to be different from in the physical schema. This data is used to test if a virtual column will be materialised so I guess it shouldn't be testing int64 -> float64 casting at the same time.

Tom-Newton · 2024-11-11T16:32:58Z

Sorry for the direct ping but @lidavidm please could you review when you get a chance. I assume you are the right person to review this?

lidavidm

Looks reasonable to me, thanks. CC @bkietz for some more eyes?

Co-authored-by: David Li <li.davidm96@gmail.com>

lidavidm · 2024-11-25T00:59:02Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

conbench-apache-arrow · 2024-11-25T07:06:42Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c14d55d.

There were 132 benchmark results with an error:

Commit Run on arm64-t4g-2xlarge-linux at 2024-11-25 02:34:37Z
- tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
- tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1
and 130 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

Tom-Newton · 2024-11-25T07:20:34Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

Thanks for reviewing 🙂

Tom-Newton · 2024-10-30T18:14:55Z

Rationale for this change

Sometimes its useful to add a column full of nulls in a cast. Currently this is supported in top level columns but not inside structs. Example where this is important: delta-io/delta-rs#1610

What changes are included in this PR?

Add support for filling in columns full of null for nullable struct fields.
I've gone for a fairly minimal change that achieves what I needed but I wonder if there should be a more significant change so that this casting is done by field name and ignore the field order.

Are these changes tested?

Yes. The expected behaviour in several existing tests has been altered and a couple of new tests have been added.

I also rolled out a custom build with this change internally because it suddenly became a critical problem.

Are there any user-facing changes?

Yes. There are scenarios that previously failed with struct fields don't match or are in the wrong order but now succeed after filling in nulls.

GitHub Issue: [C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555

github-actions · 2024-10-30T18:15:23Z

⚠️ GitHub issue #44555has been automatically assigned in GitHub to PR creator.

Tom-Newton · 2024-10-30T18:50:17Z

I'm not confident that I've taken the right approach here but I think its ready for review.

Tom-Newton · 2024-10-30T19:49:40Z

Ok, it looks like there is at least one more tests I need to update. I can probably sort that tomorrow.

Tom-Newton · 2024-11-01T10:55:27Z

 field("c", struct_({
 field("d", int64()),
- field("e", float64()),
+ field("e", int64()),


I suspect this was not supposed to be different from in the physical schema. This data is used to test if a virtual column will be materialised so I guess it shouldn't be testing int64 -> float64 casting at the same time.

Tom-Newton · 2024-11-11T16:32:58Z

Sorry for the direct ping but @lidavidm please could you review when you get a chance. I assume you are the right person to review this?

lidavidm

Looks reasonable to me, thanks. CC @bkietz for some more eyes?

Co-authored-by: David Li <li.davidm96@gmail.com>

lidavidm · 2024-11-25T00:59:02Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

conbench-apache-arrow · 2024-11-25T07:06:42Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c14d55d.

There were 132 benchmark results with an error:

Commit Run on arm64-t4g-2xlarge-linux at 2024-11-25 02:34:37Z
- tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
- tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1
and 130 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

Tom-Newton · 2024-11-25T07:20:34Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

Thanks for reviewing 🙂

Tom-Newton · 2024-10-30T18:14:55Z

Rationale for this change

Sometimes its useful to add a column full of nulls in a cast. Currently this is supported in top level columns but not inside structs. Example where this is important: delta-io/delta-rs#1610

What changes are included in this PR?

Add support for filling in columns full of null for nullable struct fields.
I've gone for a fairly minimal change that achieves what I needed but I wonder if there should be a more significant change so that this casting is done by field name and ignore the field order.

Are these changes tested?

Yes. The expected behaviour in several existing tests has been altered and a couple of new tests have been added.

I also rolled out a custom build with this change internally because it suddenly became a critical problem.

Are there any user-facing changes?

Yes. There are scenarios that previously failed with struct fields don't match or are in the wrong order but now succeed after filling in nulls.

GitHub Issue: [C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555

github-actions · 2024-10-30T18:15:23Z

⚠️ GitHub issue #44555has been automatically assigned in GitHub to PR creator.

Tom-Newton · 2024-10-30T18:50:17Z

I'm not confident that I've taken the right approach here but I think its ready for review.

Tom-Newton · 2024-10-30T19:49:40Z

Ok, it looks like there is at least one more tests I need to update. I can probably sort that tomorrow.

Tom-Newton · 2024-11-01T10:55:27Z

 field("c", struct_({
 field("d", int64()),
- field("e", float64()),
+ field("e", int64()),


I suspect this was not supposed to be different from in the physical schema. This data is used to test if a virtual column will be materialised so I guess it shouldn't be testing int64 -> float64 casting at the same time.

Tom-Newton · 2024-11-11T16:32:58Z

Sorry for the direct ping but @lidavidm please could you review when you get a chance. I assume you are the right person to review this?

lidavidm

Looks reasonable to me, thanks. CC @bkietz for some more eyes?

Co-authored-by: David Li <li.davidm96@gmail.com>

lidavidm · 2024-11-25T00:59:02Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

conbench-apache-arrow · 2024-11-25T07:06:42Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c14d55d.

There were 132 benchmark results with an error:

Commit Run on arm64-t4g-2xlarge-linux at 2024-11-25 02:34:37Z
- tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
- tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1
and 130 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

Tom-Newton · 2024-11-25T07:20:34Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

Thanks for reviewing 🙂

Tom-Newton · 2024-10-30T18:14:55Z

Rationale for this change

Sometimes its useful to add a column full of nulls in a cast. Currently this is supported in top level columns but not inside structs. Example where this is important: delta-io/delta-rs#1610

What changes are included in this PR?

Add support for filling in columns full of null for nullable struct fields.
I've gone for a fairly minimal change that achieves what I needed but I wonder if there should be a more significant change so that this casting is done by field name and ignore the field order.

Are these changes tested?

Yes. The expected behaviour in several existing tests has been altered and a couple of new tests have been added.

I also rolled out a custom build with this change internally because it suddenly became a critical problem.

Are there any user-facing changes?

Yes. There are scenarios that previously failed with struct fields don't match or are in the wrong order but now succeed after filling in nulls.

GitHub Issue: [C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555

github-actions · 2024-10-30T18:15:23Z

⚠️ GitHub issue #44555has been automatically assigned in GitHub to PR creator.

Tom-Newton · 2024-10-30T18:50:17Z

I'm not confident that I've taken the right approach here but I think its ready for review.

Tom-Newton · 2024-10-30T19:49:40Z

Ok, it looks like there is at least one more tests I need to update. I can probably sort that tomorrow.

Tom-Newton · 2024-11-01T10:55:27Z

 field("c", struct_({
 field("d", int64()),
- field("e", float64()),
+ field("e", int64()),


I suspect this was not supposed to be different from in the physical schema. This data is used to test if a virtual column will be materialised so I guess it shouldn't be testing int64 -> float64 casting at the same time.

Tom-Newton · 2024-11-11T16:32:58Z

Sorry for the direct ping but @lidavidm please could you review when you get a chance. I assume you are the right person to review this?

lidavidm

Looks reasonable to me, thanks. CC @bkietz for some more eyes?

Co-authored-by: David Li <li.davidm96@gmail.com>

lidavidm · 2024-11-25T00:59:02Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

conbench-apache-arrow · 2024-11-25T07:06:42Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c14d55d.

There were 132 benchmark results with an error:

Commit Run on arm64-t4g-2xlarge-linux at 2024-11-25 02:34:37Z
- tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
- tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1
and 130 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

Tom-Newton · 2024-11-25T07:20:34Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

Thanks for reviewing 🙂

Tom-Newton · 2024-10-30T18:14:55Z

Rationale for this change

Sometimes its useful to add a column full of nulls in a cast. Currently this is supported in top level columns but not inside structs. Example where this is important: delta-io/delta-rs#1610

What changes are included in this PR?

Add support for filling in columns full of null for nullable struct fields.
I've gone for a fairly minimal change that achieves what I needed but I wonder if there should be a more significant change so that this casting is done by field name and ignore the field order.

Are these changes tested?

Yes. The expected behaviour in several existing tests has been altered and a couple of new tests have been added.

I also rolled out a custom build with this change internally because it suddenly became a critical problem.

Are there any user-facing changes?

Yes. There are scenarios that previously failed with struct fields don't match or are in the wrong order but now succeed after filling in nulls.

GitHub Issue: [C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555

github-actions · 2024-10-30T18:15:23Z

⚠️ GitHub issue #44555has been automatically assigned in GitHub to PR creator.

Tom-Newton · 2024-10-30T18:50:17Z

I'm not confident that I've taken the right approach here but I think its ready for review.

Tom-Newton · 2024-10-30T19:49:40Z

Ok, it looks like there is at least one more tests I need to update. I can probably sort that tomorrow.

Tom-Newton · 2024-11-01T10:55:27Z

 field("c", struct_({
 field("d", int64()),
- field("e", float64()),
+ field("e", int64()),


I suspect this was not supposed to be different from in the physical schema. This data is used to test if a virtual column will be materialised so I guess it shouldn't be testing int64 -> float64 casting at the same time.

Tom-Newton · 2024-11-11T16:32:58Z

Sorry for the direct ping but @lidavidm please could you review when you get a chance. I assume you are the right person to review this?

lidavidm

Looks reasonable to me, thanks. CC @bkietz for some more eyes?

Co-authored-by: David Li <li.davidm96@gmail.com>

lidavidm · 2024-11-25T00:59:02Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

conbench-apache-arrow · 2024-11-25T07:06:42Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c14d55d.

There were 132 benchmark results with an error:

Commit Run on arm64-t4g-2xlarge-linux at 2024-11-25 02:34:37Z
- tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
- tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1
and 130 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

Tom-Newton · 2024-11-25T07:20:34Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

Thanks for reviewing 🙂

Tom-Newton · 2024-10-30T18:14:55Z

Rationale for this change

Sometimes its useful to add a column full of nulls in a cast. Currently this is supported in top level columns but not inside structs. Example where this is important: delta-io/delta-rs#1610

What changes are included in this PR?

Add support for filling in columns full of null for nullable struct fields.
I've gone for a fairly minimal change that achieves what I needed but I wonder if there should be a more significant change so that this casting is done by field name and ignore the field order.

Are these changes tested?

Yes. The expected behaviour in several existing tests has been altered and a couple of new tests have been added.

I also rolled out a custom build with this change internally because it suddenly became a critical problem.

Are there any user-facing changes?

Yes. There are scenarios that previously failed with struct fields don't match or are in the wrong order but now succeed after filling in nulls.

GitHub Issue: [C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555

github-actions · 2024-10-30T18:15:23Z

⚠️ GitHub issue #44555has been automatically assigned in GitHub to PR creator.

Tom-Newton · 2024-10-30T18:50:17Z

I'm not confident that I've taken the right approach here but I think its ready for review.

Tom-Newton · 2024-10-30T19:49:40Z

Ok, it looks like there is at least one more tests I need to update. I can probably sort that tomorrow.

Tom-Newton · 2024-11-01T10:55:27Z

 field("c", struct_({
 field("d", int64()),
- field("e", float64()),
+ field("e", int64()),


I suspect this was not supposed to be different from in the physical schema. This data is used to test if a virtual column will be materialised so I guess it shouldn't be testing int64 -> float64 casting at the same time.

Tom-Newton · 2024-11-11T16:32:58Z

Sorry for the direct ping but @lidavidm please could you review when you get a chance. I assume you are the right person to review this?

lidavidm

Looks reasonable to me, thanks. CC @bkietz for some more eyes?

Co-authored-by: David Li <li.davidm96@gmail.com>

lidavidm · 2024-11-25T00:59:02Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

conbench-apache-arrow · 2024-11-25T07:06:42Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c14d55d.

There were 132 benchmark results with an error:

Commit Run on arm64-t4g-2xlarge-linux at 2024-11-25 02:34:37Z
- tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
- tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1
and 130 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

Tom-Newton · 2024-11-25T07:20:34Z

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

Thanks for reviewing 🙂

Tom-Newton added 13 commits October 28, 2024 19:47

Maybe working
5111a55

Satisfactory tests
6f44e37

Tidy
1f3a07d

Improve implementation
cec4090

Keep failures on out of order
b5f97b4

Fix some tests
febd731

All tests pass
70cc216

More test cases
f5d08c1

New tests pass
d886756

REduce duplication
c89f1ec

Tidy
9d5d436

Update comment
3f40980

Revert debug change
0cacea9

github-actionsBot added Component: C++ awaiting review Awaiting review labels Oct 30, 2024

Tom-Newton added 4 commits October 30, 2024 18:41

Tidy
3377c70

Add some extra assertions with non nullable fields
be084cd

Tidy test numbering
7feb26e

autoformat
680f224

Tom-Newton marked this pull request as ready for review October 30, 2024 18:50

Tom-Newton added 6 commits October 31, 2024 09:58

Some progress on MaterializationOfNestedVirtualColumn test
358822f

First test pass
378cef1

I think its very close to working
10c45a1

Even closer
71f9258

Success
69d962e

REvert debug changes
ca0285c

Tom-Newton commented Nov 1, 2024
View reviewed changes

Autoformat
583701d

lidavidm reviewed Nov 14, 2024
View reviewed changes

Comment threadcpp/src/arrow/compute/kernels/scalar_cast_nested.cc Outdated
Comment threadcpp/src/arrow/compute/kernels/scalar_cast_nested.cc Outdated

github-actionsBot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Nov 14, 2024

Update cpp/src/arrow/compute/kernels/scalar_cast_nested.cc
f41a941
Co-authored-by: David Li <li.davidm96@gmail.com>

github-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Nov 14, 2024

Add kFillNullSentinel
13dc2d1

github-actionsBot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Nov 14, 2024

Move kFillNullSentinel
dc65397

github-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Nov 14, 2024

lidavidm approved these changes Nov 21, 2024
View reviewed changes

github-actionsBot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Nov 21, 2024

lidavidm merged commit c14d55d into apache:mainNov 25, 2024

lidavidm removed the awaiting merge Awaiting merge label Nov 25, 2024

lidavidm mentioned this pull request Nov 25, 2024
[C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555
Closed

Tom-Newton mentioned this pull request Nov 26, 2024
Pyarrow dataset scan can't handle schema evolved nested structs delta-io/delta-rs#1610
Closed

Tom-Newton mentioned this pull request Jan 13, 2025
[C++][Compute] Support casting struct fields to a different order #45028
Closed

ion-elgreco mentioned this pull request Mar 26, 2025
Schema Mode merge with subfields 'struct fields don't match' delta-io/delta-rs#3350
Closed

Tom-Newton mentioned this pull request Apr 23, 2025
GH-45028: [C++][Compute] Allow cast to reorder struct fields #45246
Merged