Skip to content

[core] Encapsulate split serialization protocols - #9316

Merged
JingsongLi merged 1 commit into
apache:masterfrom
leaves12138:codex/encapsulate-split-serialization
Aug 20, 2026
Merged

[core] Encapsulate split serialization protocols#9316
JingsongLi merged 1 commit into
apache:masterfrom
leaves12138:codex/encapsulate-split-serialization

Conversation

@leaves12138

Copy link
Copy Markdown
Contributor

What

  • Keep SplitSerializer focused on the outer frame, stable type ids, and type dispatch.
  • Move IncrementalSplit binary serialization into public serialize(DataOutputView) and deserialize(DataInputView) methods, matching DataSplit, ChainSplit, and IndexedSplit.
  • Let FallbackSplitImpl own its inner isFallback + wrapped split protocol.
  • Let QueryAuthSplit own its wrapped split and authorization-result protocol.
  • Keep Java writeObject / readObject on the complete outer frame for wrapper splits, avoiding recursive delegation while preserving their Java serialization layout.

Why

SplitSerializer previously duplicated the field-level formats of several split implementations. Wrapper splits also delegated their own serialize methods back to SplitSerializer, which then called split-specific helper methods.

Giving each split ownership of its inner protocol removes the circular delegation and keeps format evolution next to the corresponding data structure. SplitSerializer now only frames and dispatches split types.

Compatibility

  • SplitSerializer.VERSION remains 1.
  • IncrementalSplit.VERSION remains 1.
  • The Java serialization body of IncrementalSplit is unchanged; its existing logic is extracted into reusable methods.
  • Wrapper Java serialization continues to use the complete SplitSerializer frame.
  • The outer Fallback and QueryAuth split bytes are unchanged.
  • split-v1-incremental is refreshed because the outer protocol now delegates to the versioned inner IncrementalSplit payload.

Scope

This PR contains only the split serialization refactor. It does not include DataFileMeta schema or column sequence changes from #9313.

Tests

  • SplitSerializerTest
  • QueryAuthSplitTest
  • Flink PendingSplitsCheckpointSerializerTest
  • Core Spotless and Checkstyle

@leaves12138
leaves12138 marked this pull request as ready for review August 20, 2026 07:55
@leaves12138
leaves12138 marked this pull request as draft August 20, 2026 08:01
@leaves12138
leaves12138 marked this pull request as ready for review August 20, 2026 08:03

@JingsongLiJingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit e5265e7 into apache:masterAug 20, 2026
13 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@leaves12138@JingsongLi