GH-50623: [C++][Python] Fix IPC serialization for extension storage layouts

[fallenmi]

Rationale for this change

The IPC writer selected the top-level validity-buffer layout from the logical extension type. Extension arrays backed by null or union storage therefore received an extra validity-buffer entry, while the reader reconstructed the physical storage layout without that entry. The resulting buffer shift produced invalid data and could lead to a crash when values were accessed.

Fixes#50623.
Fixes#15068.

What changes are included in this PR?

• Select the IPC validity-buffer layout from DataType::storage_id(), which is unchanged for ordinary Arrow types and resolves to the physical type for extensions.
• Add roundtrip regressions for extension arrays backed by dense union, sparse union, and null storage.
• Exercise both IPC metadata versions V4 and V5. The null-storage test includes an adjacent int64 column so a shifted buffer is detected directly.

Are these changes tested?

• On unpatched current main, both new regressions failed with Buffer #1 too small validation errors.
• Focused patched regressions: 2/2 passed.
• Full arrow-ipc-read-write-test: 412/412 passed with the official arrow-testing data.
• clang-format 20.1.8 dry run and git diff --check: passed.

Are there any user-facing changes?

Yes. IPC file and stream roundtrips for extension types with null or union storage now preserve valid buffer layouts. There is no public API or ABI change.

This PR contains a Critical Fix. It prevents the IPC writer from producing invalid data for valid extension arrays and addresses the crash path described in #50623.

AI-assisted contribution disclosure

OpenAI Codex generated the patch and regression tests under the account owner authorization. Codex reproduced the issue on current main, reviewed the complete diff, audited visible ownership and overlap, and ran the validation listed above. This PR remains a Draft pending the account owner personal review before it is marked ready.

• GitHub Issue: [C++][Python] IPC serialization drops a union's buffers when the union is wrapped in an extension type (invalid data; segfault on read) #50623

[github-actions]

⚠️ GitHub issue #50623has been automatically assigned in GitHub to PR creator.

[fallenmi]

Merged PR #50927 supersedes this draft by unwrapping each extension array to its physical storage before validity-buffer, layout, and type dispatch. That covers the union cases and the same null-storage path without carrying a second conflicting implementation. Closing to avoid duplicate work; #15068 can remain the place for any explicit null-storage regression follow-up. Codex assisted with this status check; the account owner authorized the closure.