Skip to content

[SPARK-45070][SQL][DOCS] Describe the binary and datetime formats of to_char/to_varchar - #42801

Closed
MaxGekk wants to merge 6 commits into
apache:masterfrom
MaxGekk:doc-to_char-api
Closed

[SPARK-45070][SQL][DOCS] Describe the binary and datetime formats of to_char/to_varchar#42801
MaxGekk wants to merge 6 commits into
apache:masterfrom
MaxGekk:doc-to_char-api

Conversation

@MaxGekk

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

In the PR, I propose to document the recent changes related to the format of the to_char/to_varchar functions:

  1. binary formats added by [SPARK-44983][SQL] Convert binary to string by to_char for the formats: hex, base64, utf-8 #42632
  2. datetime formats introduced by [SPARK-44868][SQL] Convert datetime to string by to_char/to_varchar #42534

Why are the changes needed?

To inform users about recent changes.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

By CI.

Was this patch authored or co-authored using generative AI tooling?

No.

@MaxGekkMaxGekk changed the title [WIP][SQL][DOCS] Describe the binary and datetime formats of to_char/to_varchar[SPARK-45070][SQL][DOCS] Describe the binary and datetime formats of to_char/to_varcharSep 4, 2023
@MaxGekk
MaxGekk marked this pull request as ready for review September 4, 2023 13:20
Comment threadsql/core/src/main/scala/org/apache/spark/sql/functions.scala

@dongjoon-hyundongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.

* <li>'base64': a base 64 string.</li>
* <li>'hex': a string in the hexadecimal format.</li>
* <li>'utf-8': the input binary is decoded to UTF-8 string.</li>
* </ul>

@dongjoon-hyundongjoon-hyunSep 4, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To @MaxGekk , it seems our Linter CI doesn't allow this kind of human-friendly formatting in the comment.

The scalafmt check failed on connector/connect at following occurrences:
Requires formatting: functions.scala
Before submitting your change, please make sure to format your code using the following command:
./build/mvn -Pscala-2.12 scalafmt:format -Dscalafmt.skip=false -Dscalafmt.validateOnly=false -Dscalafmt.changedOnly=false -pl connector/connect/common -pl connector/connect/server -pl connector/connect/client/jvm
Error: Process completed with exit code 1.

The recommended style is something like the following. Please fix the style by run the above command line.

- * brackets if the input value is negative.</li> </ul>
- * If `e` is a datetime, `format` shall be a valid datetime pattern, see
- * <a href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html">Datetime Patterns</a>.
- * If `e` is a binary, it is converted to a string in one of the formats:
- * <ul>
- * <li>'base64': a base 64 string.</li>
- * <li>'hex': a string in the hexadecimal format.</li>
- * <li>'utf-8': the input binary is decoded to UTF-8 string.</li>
- * </ul>
+ * brackets if the input value is negative.</li> </ul> If `e` is a datetime, `format` shall be
+ * a valid datetime pattern, see <a
+ * href="https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html">Datetime
+ * Patterns</a>. If `e` is a binary, it is converted to a string in one of the formats: <ul>
+ * <li>'base64': a base 64 string.</li> <li>'hex': a string in the hexadecimal format.</li>
+ * <li>'utf-8': the input binary is decoded to UTF-8 string.</li> </ul>

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @dongjoon-hyun

@MaxGekk

Copy link
Copy Markdown
MemberAuthor

Merging to master. Thank you, @dongjoon-hyun and @cloud-fan for review.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@MaxGekk@cloud-fan@dongjoon-hyun