Uh oh!
There was an error while loading. Please reload this page.
Fix extension type metadata propagation through casts - #22162
Fix extension type metadata propagation through casts#22162paleolimbot wants to merge 8 commits into
Conversation
| } | ||
| let can_build_cast = if requires_nested_struct_cast(&expr_type, cast_type) { | ||
| let can_build_cast = if target_field.extension_type_name().is_some() { |
There was a problem hiding this comment.
Instead of false, can this block use the passed-down CastExtension to perform a real runtime check? or is that unnecessary?
There was a problem hiding this comment.
I think this will be done in a follow-up PR that introduces the actual casting support in the extension type registry.
tschwarzinger
commented
May 22, 2026
Thank you for your work on this. To my limited understanding of the casting system, the changes look reasonable. If I can help help you somehow with prototyping or fixing bugs, lmk. |
paleolimbot
commented
May 22, 2026
Thanks...I've had trouble finding the time to finish this. I hope I can get a minimal fix in the next day or two and then focus on the |
paleolimbot
commented
Jun 24, 2026
Closing in favour of a fresh/rebased PR: #23169 |
Which issue does this PR close?
Rationale for this change
The logical
Expr::CastandExpr::TryCasthave aFieldReftarget that was added in #18136 so that logical casts can express a cast to an extension type. In combination with a SQL type planner ( #20676 ) and an optimizer rule, this enabled casting to/from extension types with custom semantics to actually occur.The ability to do this was reverted by #20836 (which removed the original test) and I am not sure that ability ever made it into a release.
What changes are included in this PR?
This PR strips specific metadata keys (extension name and extension metadata) when propagating metadata from the source of a cast to the target (because doing so may result in an invalid destination field that consumers could reject), and propagates all metadata from the (logical) cast target field (e.g., so that a cast to an extension type represented by the cast target field will have a
to_field()that communicates the extension type).For the physical cast, this PR strips the extension name and metadata keys from the source field for the default cast (i.e., where the target field of the physical cast is just a DataType). This is needed so that the logical and physical behaviour agrees. The physical cast's target field comes from the logical cast's target field, so the extra metadata added by the logical cast field is already there.
In the planner, I re-added the behaviour where a cast to an extension type is rejected with an error. Casting to an extension type can be implemented with an optimizer rule, planner, or by the mechanism I have in the works in #21071 .
I would prefer to strip metadata across a cast (as we do for scalar function calls) but released DataFusions all currently do this and so this workaround is perhaps less disruptive.
Are these changes tested?
Yes
Are there any user-facing changes?
It was in practice not common to create a
Expr::Castwith field metadata internally and thus I don't think users will see metadata changes from the inclusion of metadata from the target field. I would be surprised if stripping the extension name/metadata from the source was disruptive (it was more likely to have caused errors).