To support the "Writing and Reading Streams" section of the vignette, perhaps we should rely more on the RecordBatchStreamWriter class and less the write_record_batch function.
We should be able to write code resembling the python api :
batch<-...sink<- buffer_output_stream()
writer<- record_batch_stream_writer(sink, batch$schema())
writer$write_batch()
writer$close()
sink$getvalue()
Most of the code is there, but we need to add
RecordBatchStreamWriter$write_batch() : write a record batch to the stream. We already have RecordBatchStreamWriter$WriteRecordBatch
RecordBatchStreamWriter$close() : not sure why it is lower case close() in python but upper case in C++. We already have RecordBatchWriter$Close()
BufferOutputStream$getvalue() : we already have BufferOutputStream$Finish()
Currently the constructor for a BufferOutputStream is buffer_output_stream(), perhaps we can align with python and make it BufferOutputStream, that would not clash with the arrow::BufferOutputStream class because of the namespacing.
Reporter: Romain Francois / @romainfrancois
Assignee: Romain Francois / @romainfrancois
PRs and other links:
Note: This issue was originally created as ARROW-3842. Please see the migration documentation for further details.
To support the "Writing and Reading Streams" section of the vignette, perhaps we should rely more on the RecordBatchStreamWriter class and less the
write_record_batchfunction.We should be able to write code resembling the python api :
Most of the code is there, but we need to add
RecordBatchStreamWriter$write_batch() : write a record batch to the stream. We already have RecordBatchStreamWriter$WriteRecordBatch
RecordBatchStreamWriter$close() : not sure why it is lower case close() in python but upper case in C++. We already have RecordBatchWriter$Close()
BufferOutputStream$getvalue() : we already have BufferOutputStream$Finish()
Currently the constructor for a BufferOutputStream is buffer_output_stream(), perhaps we can align with python and make it BufferOutputStream, that would not clash with the
arrow::BufferOutputStreamclass because of the namespacing.Reporter: Romain Francois / @romainfrancois
Assignee: Romain Francois / @romainfrancois
PRs and other links:
Note: This issue was originally created as ARROW-3842. Please see the migration documentation for further details.