[SPARK-57999][SQL][PYTHON][DOCS] Document accepted input and return types for built-in functions

[HyukjinKwon]

What changes were proposed in this pull request?

This PR documents, for every built-in function argument, which input types it accepts, and for
the DataFrame APIs, what type the function returns — across the three public documentation
surfaces:

• SQL @ExpressionDescriptionarguments blocks (the SQL functions reference page),
• Scala functions.scala Scaladoc (@param / @return),
• PySpark builtin.py docstrings (Parameters / Returns).

Each argument names its accepted type, e.g. SQL An expression that evaluates to an integer. /
DataFrame A column that evaluates to an integer.. The wording adapts to the argument kind:

• Value arguments name their accepted type(s), e.g. a numeric, a string or binary,
  a numeric, interval, date, timestamp, or time.
• Constant-only arguments are marked Must be a constant. (e.g. ceil/floor's scale).
• Lambda arguments of higher-order functions are described as lambdas, e.g. A lambda function..

Return types are documented on the DataFrame surfaces as, e.g., Returns a column that evaluates to a double., Returns a column of the same type as the input. (e.g. abs, coalesce, greatest),
or Returns a column of the element type of the input array, or the value type of the input map.
for collection accessors (e.g. element_at, explode).

Conventions used for the documented types:

• Numeric umbrella. Where numeric subtypes are accepted via implicit cast (e.g. the statistical
  aggregates corr/covar_*/stddev_*/var_*/regr_*, or divide), the argument is documented
  as a numeric (or a numeric or interval) rather than enumerating double/decimal/long,
  matching how mainstream SQL references document argument types. Genuinely narrow sets are kept
  (e.g. an integral, an integral, decimal, or interval for div).
• TIMESTAMP_NTZ is documented as timestamp in type phrases (the distinction is kept in
  function names and examples, not in user-facing type descriptions).

This PR contains only documentation text. It has no functional or API change.

Why are the changes needed?

Users of the SQL reference, the Scala functions API, and the PySpark functions API previously
had no per-argument type information for most built-in functions, and no documented return type on
the DataFrame APIs. This PR fills that gap consistently across all three surfaces.

How was this patch generated and verified?

The documented types were derived from the analyzer's actual behavior, then adjusted to the
conventions above and cross-checked against each expression's inputTypes / checkInputDataTypes
/ dataType and the ExpressionsSchemaSuite output schema:

1. A probe ran every function/argument position through the analyzer, recording which types are
   accepted (natively or via cast), rejected, or constant-only, and measured each function's output
   type (including functions whose return type follows their input or an input's element/value type).
2. The measured types were rendered into the three surfaces, applying the numeric-umbrella and
   TIMESTAMP_NTZ conventions.
3. Accepted/return types were then verified against Catalyst source, correcting cases the raw probe
   got wrong — e.g. to_char/to_varchar accept numeric, date, timestamp, time, or binary;
   divide dividend is numeric or interval and divisor numeric; array element-args
   (array_contains/append/prepend/position/remove/insert) take the array's element type;
   element_at/explode document the map case; unix_nanos returns a decimal.

ExpressionInfoSuite, the SPARK-32870 argument-format tests, the docs build, Scala style/format
(scalastyle + scalafmt), and Python lint (flake8 + black) all pass.

Does this PR introduce any user-facing change?

Yes — documentation only. The SQL built-in function reference, Scala functions Scaladoc, and
PySpark functions docstrings now state each argument's accepted input type and each DataFrame
function's return type. There is no behavioral or API change.

How was this patch tested?

Existing documentation and expression tests (ExpressionInfoSuite, docs build) pass. The
documentation text was cross-checked against the analyzer's measured accept/reject behavior and
Catalyst source for every function and argument position, as described above.

[HyukjinKwon]

I think it's quite good now. I am triggering the last multi-agent orchastrated review

[HyukjinKwon]

cc @cloud-fan too

[dongjoon-hyun]

Could you check the linter failures?

Scalastyle checks failed at following occurrences:
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:2967: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:2996: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:3864: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:3899: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:3934: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:4190: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:4212: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:4937: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:4958: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:4980: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:5001: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:5047: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:5093: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:8757: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:8884: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:8935: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:9355: File line length exceeds 100 characters
[error] /__w/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/functions.scala:9408: File line length exceeds 100 characters

[dongjoon-hyun]

If you don't mind, shall we split this into more smaller reviewable PRs? For example, at least two; one for SQL, one for Python, @HyukjinKwon ? Although we live in agentic era, the PR should be human-reviewable as much as possible.

[HyukjinKwon]

Yeah will do

[HyukjinKwon]

Superseded by smaller, per-surface PRs for reviewability (per @dongjoon-hyun's suggestion). SPARK-57999 is now an umbrella, with one sub-task and PR per documentation surface:

• SQL (@ExpressionDescription reference): SPARK-58085 — [SPARK-58085][SQL][DOCS] Document accepted input types for built-in functions in the SQL function reference #57183
• Scala (functions.scala): SPARK-58086 — [SPARK-58086][SQL][DOCS] Document accepted input and return types in the Scala functions API #57184
• PySpark (builtin.py): SPARK-58087 — [SPARK-58087][PYTHON][DOCS] Document accepted input and return types in the PySpark functions API #57185

Each carries the exact reviewed content from this PR for its surface (scalastyle line-length fixes and the schema_of_json return-type fix included). Closing this in favor of the three above.

[dongjoon-hyun]

Thank you for the decision, @HyukjinKwon !