Uh oh!
There was an error while loading. Please reload this page.
fix: preserve ProjectionExec metadata during serialization - #25009
Open
goutamadwant wants to merge 1 commit into
Open
fix: preserve ProjectionExec metadata during serialization#25009goutamadwant wants to merge 1 commit into
goutamadwant wants to merge 1 commit into
Conversation
Thank you for opening this pull request! Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch). Details |
codecov-commenter
commented
Sep 7, 2026
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@## main #25009 +/- ##
==========================================
+ Coverage 81.69% 81.75% +0.05%
==========================================
Files 1127 1127 Lines 415471 415680 +209 Branches 415471 415680 +209 ==========================================
+ Hits 339424 339844 +420 + Misses 56108 55850 -258 - Partials 19939 19986 +47 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
gene-bordegaray
left a comment
Contributor
There was a problem hiding this comment.
One concern, other than that thank you 👍
| let input = ctx.encode_child(input)?; | ||
| let expr = ctx.encode_expressions(projection_exprs.iter().map(|p| &p.expr))?; | ||
| let expr_name = projection_exprs.iter().map(|p| p.alias.clone()).collect(); | ||
| let schema = if *overrides_metadata { |
Contributor
There was a problem hiding this comment.
This conditioins shouldnt just be if we are overriding metadata I believe. Rather we should be checking if we have metadata in general.
| )); | ||
| let plan = Arc::new(ProjectionExec::try_new( | ||
| vec![(col("value", &input_schema)?, "value".to_string())], | ||
| Arc::new(EmptyExec::new(Arc::clone(&input_schema))), |
Contributor
There was a problem hiding this comment.
this is preserving the metadata through the child after decoding, not a proojection that needs to rederive the metadata completely from itself.
Could we add a test that forces the projection to completely rederive the metadata from its own proto after roundtrip 👍
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
Physical-plan serialization loses projection field and schema metadata when it differs from metadata derived from the input and expressions.
What changes are included in this PR?
ProjectionExecNode, emitted only when the projection overrides metadata.try_new_with_schema_metadata, keeping field names, types, and nullability derived from the expressions.What is the testing strategy for this PR?
roundtrip_projection_metadatareproduces the loss before the fix and passes afterward.decode_projection_schema_only_replaces_metadatachecks expression-derived field properties and invalid field counts.Are there any user-facing changes?
Projection metadata survives physical-plan serialization. The generated
ProjectionExecNodeRust struct gains an optionalschemafield. New readers continue to accept older payloads. This targets main, not a 55.1 backport.