[HADOOP-19097] core-default fs.s3a.connection.establish.timeout value too low -warning always printed - ASF JIRA

XML

Word

Printable

JSON

in core-default we set the value of fs.s3a.connection.establish.timeout to 5s

<property>
  <name>fs.s3a.connection.establish.timeout</name>
  <value>5s</value>
</property>

but there is a minimum of 15s, so this prints a warning

2024-02-29 10:39:27,369 WARN impl.ConfigurationHelper: Option fs.s3a.connection.establish.timeout is too low (5,000 ms). Setting to 15,000 ms instead

is related to

SPARK-47552 Set spark.hadoop.fs.s3a.connection.establish.timeout to 30s

links to

GitHub Pull Request #6601