[SPARK-47552][CORE][FOLLOWUP] Set spark.hadoop.fs.s3a.connection.establish.timeout to numeric

[cloud-fan]

What changes were proposed in this pull request?

This is a followup of #45710 . Some custom FileSystem implementations read the hadoop.fs.s3a.connection.establish.timeout config as numeric, and do not support the 30s syntax. To make it safe, this PR proposes to set this conf to 30000 instead of 30s. I checked the doc page and this config is milliseconds.

Why are the changes needed?

more compatible with custom FileSystem implementations.

Does this PR introduce any user-facing change?

no

How was this patch tested?

manual

Was this patch authored or co-authored using generative AI tooling?

no

[cloud-fan]

cc @dongjoon-hyun

[yaooqinn]

It looks more reasonable to use setIfMissing with hadoop.Configuration, #45710 seems to force overriding custom value of s3a.connection.establish.timeout if it's missing in spark conf.

[cloud-fan]

@yaooqinn It still uses setIfMissing, or did I miss something?

[yaooqinn]

It's irrelevant to your PR. I mean #45710 brought this behavioral change. If a user defines s3a.connection.establish.timeout=100s in hdfs-site.xml, then he/she actually gets 30s.

[cloud-fan]

also cc @viirya

[viirya]

For the purpose of this follow up, it looks good.

For the question, if spark.hadoop.fs.s3a.connection.establish.timeout is set and s3a.connection.establish.timeout is also set, which one will be used?

[cloud-fan]

let me merge this follow-up first. We can continue the discussion in the original PR.