GH-32190: [C++][Compute] Implement cumulative prod, max and min functions

[js8544]

Rationale for this change

Implement cumulative prod, max and min compute functions

What changes are included in this PR?

1. Add implementations, docs and tests for the three functions.
2. Refactor CumulativeSumOptions to CumulativeOptions for reusability.
3. Fix a bug where GenericFromScalar(GenericToScalar(std::nullopt)) != std::nullopt.
4. Remove an unnecessary Cast with the default start value.
5. Add tests to check behavior with NaN.

I'll explain some of the changes in comments.

Are these changes tested?

Yes, in vector_accumulative_ops_test.cc and test_compute.py

Are there any user-facing changes?

No. The data members of CumulativeSumOptions are changed, but the member functions behave as before. And std::optional also can be constructed directly from T. So users should not feel any difference.

• Closes: [C++] Implement cumulative product, max, and min compute functions #32190

[github-actions]

⚠️ GitHub issue #32190has been automatically assigned in GitHub to PR creator.

[bkietz]

Thanks for doing this!

This looks good, a few minor comments:

[bkietz]

@jorisvandenbossche Does this need an alias?

Suggested change

	CumulativeOptions,
	CumulativeOptions,
	CumulativeOptionsasCumulativeSumOptions,

[jorisvandenbossche]

For backwards compatibility, that's probably useful, yes (also on the C++ side this was done).

(although we should maybe actually add it as its own class and deprecate it, so we can remove the alias at some point. But this can also be done as a follow-up later)

[bkietz]

Follow up issue here #36240

[bkietz]

These cumulative ops aren't currently instantiated for decimal types anyway: https://github.com/apache/arrow/blob/44edc27/cpp/src/arrow/compute/kernels/codegen_internal.h#L1070-L1100

... if you feel like filing a follow up issue for them, it should be fairly straightforward to specify these as specializations:

Suggested change

	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<>
	static constexpr Decimal128value<Decimal128>=Decimal128::GetMinSentinel();

Multiplication will require more effort since we can't construct 1 without knowing the scale.

[js8544]

I'll remove them for this pr. Will send another pr for decimals.

[bkietz]

LGTM, thanks!

[bkietz]

CI failure is a backpressure test flake-out in AsOfJoin, unrelated.

[icexelloss]

Thanks @bkietz - created #36248 to track the Asof Join backpresure test issue

[conbench-apache-arrow]

Conbench analyzed the 6 benchmark runs on commit e3eb5898.

There were 6 benchmark results indicating a performance regression:

• Commit Run on ursa-thinkcentre-m75q at 2023-06-25 03:03:51Z
  • params=1048576/1, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
  • params=/1048576/0, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
• and 4 more (see the report linked below)

The full Conbench report has more details.

[js8544]

Rationale for this change

Implement cumulative prod, max and min compute functions

What changes are included in this PR?

1. Add implementations, docs and tests for the three functions.
2. Refactor CumulativeSumOptions to CumulativeOptions for reusability.
3. Fix a bug where GenericFromScalar(GenericToScalar(std::nullopt)) != std::nullopt.
4. Remove an unnecessary Cast with the default start value.
5. Add tests to check behavior with NaN.

I'll explain some of the changes in comments.

Are these changes tested?

Yes, in vector_accumulative_ops_test.cc and test_compute.py

Are there any user-facing changes?

No. The data members of CumulativeSumOptions are changed, but the member functions behave as before. And std::optional also can be constructed directly from T. So users should not feel any difference.

• Closes: [C++] Implement cumulative product, max, and min compute functions #32190

[github-actions]

⚠️ GitHub issue #32190has been automatically assigned in GitHub to PR creator.

[bkietz]

Thanks for doing this!

This looks good, a few minor comments:

[bkietz]

@jorisvandenbossche Does this need an alias?

Suggested change

	CumulativeOptions,
	CumulativeOptions,
	CumulativeOptionsasCumulativeSumOptions,

[jorisvandenbossche]

For backwards compatibility, that's probably useful, yes (also on the C++ side this was done).

(although we should maybe actually add it as its own class and deprecate it, so we can remove the alias at some point. But this can also be done as a follow-up later)

[bkietz]

Follow up issue here #36240

[bkietz]

These cumulative ops aren't currently instantiated for decimal types anyway: https://github.com/apache/arrow/blob/44edc27/cpp/src/arrow/compute/kernels/codegen_internal.h#L1070-L1100

... if you feel like filing a follow up issue for them, it should be fairly straightforward to specify these as specializations:

Suggested change

	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<>
	static constexpr Decimal128value<Decimal128>=Decimal128::GetMinSentinel();

Multiplication will require more effort since we can't construct 1 without knowing the scale.

[js8544]

I'll remove them for this pr. Will send another pr for decimals.

[bkietz]

LGTM, thanks!

[bkietz]

CI failure is a backpressure test flake-out in AsOfJoin, unrelated.

[icexelloss]

Thanks @bkietz - created #36248 to track the Asof Join backpresure test issue

[conbench-apache-arrow]

Conbench analyzed the 6 benchmark runs on commit e3eb5898.

There were 6 benchmark results indicating a performance regression:

• Commit Run on ursa-thinkcentre-m75q at 2023-06-25 03:03:51Z
  • params=1048576/1, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
  • params=/1048576/0, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
• and 4 more (see the report linked below)

The full Conbench report has more details.

[js8544]

Rationale for this change

Implement cumulative prod, max and min compute functions

What changes are included in this PR?

1. Add implementations, docs and tests for the three functions.
2. Refactor CumulativeSumOptions to CumulativeOptions for reusability.
3. Fix a bug where GenericFromScalar(GenericToScalar(std::nullopt)) != std::nullopt.
4. Remove an unnecessary Cast with the default start value.
5. Add tests to check behavior with NaN.

I'll explain some of the changes in comments.

Are these changes tested?

Yes, in vector_accumulative_ops_test.cc and test_compute.py

Are there any user-facing changes?

No. The data members of CumulativeSumOptions are changed, but the member functions behave as before. And std::optional also can be constructed directly from T. So users should not feel any difference.

• Closes: [C++] Implement cumulative product, max, and min compute functions #32190

[github-actions]

⚠️ GitHub issue #32190has been automatically assigned in GitHub to PR creator.

[bkietz]

Thanks for doing this!

This looks good, a few minor comments:

[bkietz]

@jorisvandenbossche Does this need an alias?

Suggested change

	CumulativeOptions,
	CumulativeOptions,
	CumulativeOptionsasCumulativeSumOptions,

[jorisvandenbossche]

For backwards compatibility, that's probably useful, yes (also on the C++ side this was done).

(although we should maybe actually add it as its own class and deprecate it, so we can remove the alias at some point. But this can also be done as a follow-up later)

[bkietz]

Follow up issue here #36240

[bkietz]

These cumulative ops aren't currently instantiated for decimal types anyway: https://github.com/apache/arrow/blob/44edc27/cpp/src/arrow/compute/kernels/codegen_internal.h#L1070-L1100

... if you feel like filing a follow up issue for them, it should be fairly straightforward to specify these as specializations:

Suggested change

	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<>
	static constexpr Decimal128value<Decimal128>=Decimal128::GetMinSentinel();

Multiplication will require more effort since we can't construct 1 without knowing the scale.

[js8544]

I'll remove them for this pr. Will send another pr for decimals.

[bkietz]

LGTM, thanks!

[bkietz]

CI failure is a backpressure test flake-out in AsOfJoin, unrelated.

[icexelloss]

Thanks @bkietz - created #36248 to track the Asof Join backpresure test issue

[conbench-apache-arrow]

Conbench analyzed the 6 benchmark runs on commit e3eb5898.

There were 6 benchmark results indicating a performance regression:

• Commit Run on ursa-thinkcentre-m75q at 2023-06-25 03:03:51Z
  • params=1048576/1, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
  • params=/1048576/0, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
• and 4 more (see the report linked below)

The full Conbench report has more details.

[js8544]

Rationale for this change

Implement cumulative prod, max and min compute functions

What changes are included in this PR?

1. Add implementations, docs and tests for the three functions.
2. Refactor CumulativeSumOptions to CumulativeOptions for reusability.
3. Fix a bug where GenericFromScalar(GenericToScalar(std::nullopt)) != std::nullopt.
4. Remove an unnecessary Cast with the default start value.
5. Add tests to check behavior with NaN.

I'll explain some of the changes in comments.

Are these changes tested?

Yes, in vector_accumulative_ops_test.cc and test_compute.py

Are there any user-facing changes?

No. The data members of CumulativeSumOptions are changed, but the member functions behave as before. And std::optional also can be constructed directly from T. So users should not feel any difference.

• Closes: [C++] Implement cumulative product, max, and min compute functions #32190

[github-actions]

⚠️ GitHub issue #32190has been automatically assigned in GitHub to PR creator.

[bkietz]

Thanks for doing this!

This looks good, a few minor comments:

[bkietz]

@jorisvandenbossche Does this need an alias?

Suggested change

	CumulativeOptions,
	CumulativeOptions,
	CumulativeOptionsasCumulativeSumOptions,

[jorisvandenbossche]

For backwards compatibility, that's probably useful, yes (also on the C++ side this was done).

(although we should maybe actually add it as its own class and deprecate it, so we can remove the alias at some point. But this can also be done as a follow-up later)

[bkietz]

Follow up issue here #36240

[bkietz]

These cumulative ops aren't currently instantiated for decimal types anyway: https://github.com/apache/arrow/blob/44edc27/cpp/src/arrow/compute/kernels/codegen_internal.h#L1070-L1100

... if you feel like filing a follow up issue for them, it should be fairly straightforward to specify these as specializations:

Suggested change

	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<>
	static constexpr Decimal128value<Decimal128>=Decimal128::GetMinSentinel();

Multiplication will require more effort since we can't construct 1 without knowing the scale.

[js8544]

I'll remove them for this pr. Will send another pr for decimals.

[bkietz]

LGTM, thanks!

[bkietz]

CI failure is a backpressure test flake-out in AsOfJoin, unrelated.

[icexelloss]

Thanks @bkietz - created #36248 to track the Asof Join backpresure test issue

[conbench-apache-arrow]

Conbench analyzed the 6 benchmark runs on commit e3eb5898.

There were 6 benchmark results indicating a performance regression:

• Commit Run on ursa-thinkcentre-m75q at 2023-06-25 03:03:51Z
  • params=1048576/1, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
  • params=/1048576/0, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
• and 4 more (see the report linked below)

The full Conbench report has more details.

[js8544]

Rationale for this change

Implement cumulative prod, max and min compute functions

What changes are included in this PR?

1. Add implementations, docs and tests for the three functions.
2. Refactor CumulativeSumOptions to CumulativeOptions for reusability.
3. Fix a bug where GenericFromScalar(GenericToScalar(std::nullopt)) != std::nullopt.
4. Remove an unnecessary Cast with the default start value.
5. Add tests to check behavior with NaN.

I'll explain some of the changes in comments.

Are these changes tested?

Yes, in vector_accumulative_ops_test.cc and test_compute.py

Are there any user-facing changes?

No. The data members of CumulativeSumOptions are changed, but the member functions behave as before. And std::optional also can be constructed directly from T. So users should not feel any difference.

• Closes: [C++] Implement cumulative product, max, and min compute functions #32190

[github-actions]

⚠️ GitHub issue #32190has been automatically assigned in GitHub to PR creator.

[bkietz]

Thanks for doing this!

This looks good, a few minor comments:

[bkietz]

@jorisvandenbossche Does this need an alias?

Suggested change

	CumulativeOptions,
	CumulativeOptions,
	CumulativeOptionsasCumulativeSumOptions,

[jorisvandenbossche]

For backwards compatibility, that's probably useful, yes (also on the C++ side this was done).

(although we should maybe actually add it as its own class and deprecate it, so we can remove the alias at some point. But this can also be done as a follow-up later)

[bkietz]

Follow up issue here #36240

[bkietz]

These cumulative ops aren't currently instantiated for decimal types anyway: https://github.com/apache/arrow/blob/44edc27/cpp/src/arrow/compute/kernels/codegen_internal.h#L1070-L1100

... if you feel like filing a follow up issue for them, it should be fairly straightforward to specify these as specializations:

Suggested change

	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<>
	static constexpr Decimal128value<Decimal128>=Decimal128::GetMinSentinel();

Multiplication will require more effort since we can't construct 1 without knowing the scale.

[js8544]

I'll remove them for this pr. Will send another pr for decimals.

[bkietz]

LGTM, thanks!

[bkietz]

CI failure is a backpressure test flake-out in AsOfJoin, unrelated.

[icexelloss]

Thanks @bkietz - created #36248 to track the Asof Join backpresure test issue

[conbench-apache-arrow]

Conbench analyzed the 6 benchmark runs on commit e3eb5898.

There were 6 benchmark results indicating a performance regression:

• Commit Run on ursa-thinkcentre-m75q at 2023-06-25 03:03:51Z
  • params=1048576/1, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
  • params=/1048576/0, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
• and 4 more (see the report linked below)

The full Conbench report has more details.

[js8544]

Rationale for this change

Implement cumulative prod, max and min compute functions

What changes are included in this PR?

1. Add implementations, docs and tests for the three functions.
2. Refactor CumulativeSumOptions to CumulativeOptions for reusability.
3. Fix a bug where GenericFromScalar(GenericToScalar(std::nullopt)) != std::nullopt.
4. Remove an unnecessary Cast with the default start value.
5. Add tests to check behavior with NaN.

I'll explain some of the changes in comments.

Are these changes tested?

Yes, in vector_accumulative_ops_test.cc and test_compute.py

Are there any user-facing changes?

No. The data members of CumulativeSumOptions are changed, but the member functions behave as before. And std::optional also can be constructed directly from T. So users should not feel any difference.

• Closes: [C++] Implement cumulative product, max, and min compute functions #32190

[github-actions]

⚠️ GitHub issue #32190has been automatically assigned in GitHub to PR creator.

[bkietz]

Thanks for doing this!

This looks good, a few minor comments:

[bkietz]

@jorisvandenbossche Does this need an alias?

Suggested change

	CumulativeOptions,
	CumulativeOptions,
	CumulativeOptionsasCumulativeSumOptions,

[jorisvandenbossche]

For backwards compatibility, that's probably useful, yes (also on the C++ side this was done).

(although we should maybe actually add it as its own class and deprecate it, so we can remove the alias at some point. But this can also be done as a follow-up later)

[bkietz]

Follow up issue here #36240

[bkietz]

These cumulative ops aren't currently instantiated for decimal types anyway: https://github.com/apache/arrow/blob/44edc27/cpp/src/arrow/compute/kernels/codegen_internal.h#L1070-L1100

... if you feel like filing a follow up issue for them, it should be fairly straightforward to specify these as specializations:

Suggested change

	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<>
	static constexpr Decimal128value<Decimal128>=Decimal128::GetMinSentinel();

Multiplication will require more effort since we can't construct 1 without knowing the scale.

[js8544]

I'll remove them for this pr. Will send another pr for decimals.

[bkietz]

LGTM, thanks!

[bkietz]

CI failure is a backpressure test flake-out in AsOfJoin, unrelated.

[icexelloss]

Thanks @bkietz - created #36248 to track the Asof Join backpresure test issue

[conbench-apache-arrow]

Conbench analyzed the 6 benchmark runs on commit e3eb5898.

There were 6 benchmark results indicating a performance regression:

• Commit Run on ursa-thinkcentre-m75q at 2023-06-25 03:03:51Z
  • params=1048576/1, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
  • params=/1048576/0, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
• and 4 more (see the report linked below)

The full Conbench report has more details.

[js8544]

Rationale for this change

Implement cumulative prod, max and min compute functions

What changes are included in this PR?

1. Add implementations, docs and tests for the three functions.
2. Refactor CumulativeSumOptions to CumulativeOptions for reusability.
3. Fix a bug where GenericFromScalar(GenericToScalar(std::nullopt)) != std::nullopt.
4. Remove an unnecessary Cast with the default start value.
5. Add tests to check behavior with NaN.

I'll explain some of the changes in comments.

Are these changes tested?

Yes, in vector_accumulative_ops_test.cc and test_compute.py

Are there any user-facing changes?

No. The data members of CumulativeSumOptions are changed, but the member functions behave as before. And std::optional also can be constructed directly from T. So users should not feel any difference.

• Closes: [C++] Implement cumulative product, max, and min compute functions #32190

[github-actions]

⚠️ GitHub issue #32190has been automatically assigned in GitHub to PR creator.

[bkietz]

Thanks for doing this!

This looks good, a few minor comments:

[bkietz]

@jorisvandenbossche Does this need an alias?

Suggested change

	CumulativeOptions,
	CumulativeOptions,
	CumulativeOptionsasCumulativeSumOptions,

[jorisvandenbossche]

For backwards compatibility, that's probably useful, yes (also on the C++ side this was done).

(although we should maybe actually add it as its own class and deprecate it, so we can remove the alias at some point. But this can also be done as a follow-up later)

[bkietz]

Follow up issue here #36240

[bkietz]

These cumulative ops aren't currently instantiated for decimal types anyway: https://github.com/apache/arrow/blob/44edc27/cpp/src/arrow/compute/kernels/codegen_internal.h#L1070-L1100

... if you feel like filing a follow up issue for them, it should be fairly straightforward to specify these as specializations:

Suggested change

	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<>
	static constexpr Decimal128value<Decimal128>=Decimal128::GetMinSentinel();

Multiplication will require more effort since we can't construct 1 without knowing the scale.

[js8544]

I'll remove them for this pr. Will send another pr for decimals.

[bkietz]

LGTM, thanks!

[bkietz]

CI failure is a backpressure test flake-out in AsOfJoin, unrelated.

[icexelloss]

Thanks @bkietz - created #36248 to track the Asof Join backpresure test issue

[conbench-apache-arrow]

Conbench analyzed the 6 benchmark runs on commit e3eb5898.

There were 6 benchmark results indicating a performance regression:

• Commit Run on ursa-thinkcentre-m75q at 2023-06-25 03:03:51Z
  • params=1048576/1, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
  • params=/1048576/0, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
• and 4 more (see the report linked below)

The full Conbench report has more details.

[js8544]

Rationale for this change

Implement cumulative prod, max and min compute functions

What changes are included in this PR?

1. Add implementations, docs and tests for the three functions.
2. Refactor CumulativeSumOptions to CumulativeOptions for reusability.
3. Fix a bug where GenericFromScalar(GenericToScalar(std::nullopt)) != std::nullopt.
4. Remove an unnecessary Cast with the default start value.
5. Add tests to check behavior with NaN.

I'll explain some of the changes in comments.

Are these changes tested?

Yes, in vector_accumulative_ops_test.cc and test_compute.py

Are there any user-facing changes?

No. The data members of CumulativeSumOptions are changed, but the member functions behave as before. And std::optional also can be constructed directly from T. So users should not feel any difference.

• Closes: [C++] Implement cumulative product, max, and min compute functions #32190

[github-actions]

⚠️ GitHub issue #32190has been automatically assigned in GitHub to PR creator.

[bkietz]

Thanks for doing this!

This looks good, a few minor comments:

[bkietz]

@jorisvandenbossche Does this need an alias?

Suggested change

	CumulativeOptions,
	CumulativeOptions,
	CumulativeOptionsasCumulativeSumOptions,

[jorisvandenbossche]

For backwards compatibility, that's probably useful, yes (also on the C++ side this was done).

(although we should maybe actually add it as its own class and deprecate it, so we can remove the alias at some point. But this can also be done as a follow-up later)

[bkietz]

Follow up issue here #36240

[bkietz]

These cumulative ops aren't currently instantiated for decimal types anyway: https://github.com/apache/arrow/blob/44edc27/cpp/src/arrow/compute/kernels/codegen_internal.h#L1070-L1100

... if you feel like filing a follow up issue for them, it should be fairly straightforward to specify these as specializations:

Suggested change

	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<typenameValue>
	static constexpr Valuevalue{std::numeric_limits<Value>::min()};
	template<>
	static constexpr Decimal128value<Decimal128>=Decimal128::GetMinSentinel();

Multiplication will require more effort since we can't construct 1 without knowing the scale.

[js8544]

I'll remove them for this pr. Will send another pr for decimals.

[bkietz]

LGTM, thanks!

[bkietz]

CI failure is a backpressure test flake-out in AsOfJoin, unrelated.

[icexelloss]

Thanks @bkietz - created #36248 to track the Asof Join backpresure test issue

[conbench-apache-arrow]

Conbench analyzed the 6 benchmark runs on commit e3eb5898.

There were 6 benchmark results indicating a performance regression:

• Commit Run on ursa-thinkcentre-m75q at 2023-06-25 03:03:51Z
  • params=1048576/1, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
  • params=/1048576/0, source=cpp-micro, suite=arrow-acero-aggregate-benchmark
• and 4 more (see the report linked below)

The full Conbench report has more details.