Implement initial version of to_json

[andygrove]

What is the problem the feature request solves?

Now that we have support for CreateNamedStruct in #620, we could start working on to_json functionality. This functionality is very similar to our existing logic for casting from various data types to string but with JSON formatting.

Describe the potential solution

No response

Additional context

No response

[jatin510]

Hello @andygrove I would like to work on this issue. Can you please assign it to me

[viirya]

Thanks @jatin510 . Assigned to you.

[dharanad]

@jatin510 I am willing to be a co author on this PR. Is that fine ?

Can by solving for below query ?

select to_json(named_struct(expression1_name, expression1_input[, ..., expression_n_name, expression_n_input]))

[dharanad]

@andygrove QQ: Upon checking i found out that DataFusion doesn't currently support a built-in to_json function. While implementing it directly in Comet is an option, there might be a more efficient approach by implementing the function in datafusion and reusing it here. What are your thoughts on these considerations?

[andygrove]

@andygrove QQ: Upon checking i found out that DataFusion doesn't currently support a built-in to_json function. While implementing it directly in Comet is an option, there might be a more efficient approach by implementing the function in datafusion and reusing it here. What are your thoughts on these considerations?

@dharanad It makes sense to add this in DataFusion. I see that Postgres also supports to_json.

Spark supports a number of options in to_json to control how dates and times are formatted. As long as those options are also available in the DataFusion version then I think we should be able to reuse it directly.

[andygrove]

I am working on an initial implementation of this and will have a PR up soon. This should make it easy for others to contribute to flesh out the functionality more. I will create this in Comet first since it has a lot of Spark-specific logic but maybe we can find a way to abstract that out so that we can upstream the bulk of the feature.