Uh oh!
There was an error while loading. Please reload this page.
GH-45690: [C++][Parquet] Consolidate Arrow write functions under TypedColumnWriterImpl - #45688
Conversation
TypedColumnWriterImplThere was a problem hiding this comment.
Since the implementation uses both parquet and arrow types, I thought it is better to be explicit. I can restore the previous name though.
There was a problem hiding this comment.
This makes sense and already used by some functions in this file.
There was a problem hiding this comment.
Since the implementation uses both parquet and arrow types, I thought it is better to be explicit.
+1
There was a problem hiding this comment.
Consistently using the same argument order as WriteArrow.
TypedColumnWriterImplTypedColumnWriterImplUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
This makes sense and already used by some functions in this file.
58b4427 to
93be82cCompare93be82c to
0bf997eCompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
What if array.null_count() != 0 here?
There was a problem hiding this comment.
This is the existing logic, I haven't changed it.
I there are actually null values in the array not just being nullable, then we need to pick the slower path.
There was a problem hiding this comment.
What if array.null_count() != 0 here?
It should not happen when the field is required but has null values: https://github.com/apache/arrow/blob/main/cpp/src/parquet/column_writer.cc#L1324
There was a problem hiding this comment.
Ok, thanks. Ideally we would have a DCHECK but since this is just moving code around we'll live without it.
pitrou
left a comment
There was a problem hiding this comment.
I posted some minor comments and suggestions, but this LGTM on the principle.
…terImpl This removes the need of passing the column writer instance and removes a redundant type template parameter.
0bf997e to
ad971a1CompareAfter merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 6b66c84. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 16 possible false positives for unstable benchmarks that are known to sometimes produce them. |
Rationale for this change
I am planning to introduce
WriteBatchInternalandWriteBatchSpacedInternalprivate methods in #45360 which would have required specifyingWriteArrowSerialize,WriteArrowZeroCopyandWriteTimestampsas friend functions. Then I noticed that these functions could be consolidated into the column writer making the implementation simpler.What changes are included in this PR?
WriteArrowSerialize,WriteArrowZeroCopyandWriteTimestampsto be methods onTypedColumnWriterImpl.column writerargument, reorder their parameters to align withWriteArrowpublic method.Are these changes tested?
Existing tests should cover these.
Are there any user-facing changes?
No, these are private functions and methods.
Resolves#45690
TypedColumnWriterImpl#45690