[C++][Python] Create utils for deep-copying an Array/ ArrayData

[asfimport]

Hi, I would like to request a util to deep copy an Array with the following semantic. 

Result<shared_ptr<ArrayData>> DeepCopyArrayData(const ArrayData& arr, int64 offset = 0, int64 length = -1/*, pool=...*/);

This was discussed some time back in Zulip [1]. 

[1] https://ursalabs.zulipchat.com/#narrow/stream/180245-dev/topic/create.20a.20deep.20copy.20of.20buffers

Reporter: Niranda Perera / @nirandaperera

Related issues:

• [Python] Pickling a sliced array serializes all the buffers (relates to)
• [Python] Make self_destruct and RecordBatchReader work better together (relates to)

_{Note: This issue was originally created as ARROW-14982. Please see the migration documentation for further details.}

[asfimport]

David Li / @lidavidm:
We could perhaps wrap Concatenate in a kernel and name it deep_copy. (I think Concatenate on a single array suffices to copy the data.)

[asfimport]

Antoine Pitrou / @pitrou:

with the following semantic

Which semantic exactly? I don't see a description of the expected behaviour here.

Note that several different things were discussed in Zulip. The last one was "copy only sliced buffers"...

[asfimport]

Niranda Perera / @nirandaperera:
@pitrou I thought the method signature was self-explanatory. :D Requirement is to deep copy an array from an offset (default, 0) to a length (default, length - offset). Yes, the zulip chat was about buffer offsets, but the main requirement was to deep copy arrays ultimately.

@lidavidm I am not entirely sure if we could use Concatenate (in general sense). May be, current concatenation does a deep copy even when we pass a single array vector. But, I think we could check offset=0 and return the input array (no copy) without violating the "concat" semantic IMO.

[asfimport]

Antoine Pitrou / @pitrou:
Can you explain why you have this requirement, though? I'm not saying we shouldn't add this, but I 'm a bit skeptical...

[asfimport]

Niranda Perera / @nirandaperera:
Well, its quite similar to ARROW-10739. I want to slice an array and I want the result to be independent of the parent (deep-copy slice, than a zero-copy slice). Nevertheless I think deep-copy would be a useful utility, just like concat, etc.

[asfimport]

Antoine Pitrou / @pitrou:
Well, ARROW-10739 only complains about sliced buffers. So I'm not sure I understand the requirement for a complete deep-copy.

[asfimport]

Niranda Perera / @nirandaperera:
In Cylon, we do some buffer-level in-place operations. And for those we need a deep-copy. IMO, with or without offset, the dev effort would be the same for deep-copy isn't it?

Anyways, say we have this.

def deep_copy(pa.Array: arr, int offset = 0, int length = None) -> pa.Array:

So, following cases would be equivalent. 

a = pa.Array(...)

# case 1 
c1 = deep_copy(a, offset=10)

# case 2 
b = a.slice(10)
...
c2 = deep_copy(b) # Deep copy, sees this as an array to be copied from 0 --> b.length(), but internally b has an offset from slicing 

assert(c1.data.offset = 0) # I'm not sure if ArrayData is exposed here
assert(c2.data.offset = 0) # I'm not sure if ArrayData is exposed here
assert(c1 == c2) 

[asfimport]

Antoine Pitrou / @pitrou:
Well, you can have some sliced buffers and others unsliced, for example in a struct array. And you may not always know upfront if some buffers have been sliced...

That said, let's add this and we can improve it later (for example make it more flexible).

[github-actions]

This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.