Description
Following on to SPARK-29903 and similar issues (linked), there are options available to the DataFrameReader for certain source formats, but which are not exposed properly in the relevant APIs.
These options include `timeZone` and `pathGlobFilter`. Instead of being noted under the option() method, they should be implemented directly into load APIs that support them.
Attachments
Issue Links
- relates to
-
SPARK-29903 Add documentation for recursiveFileLookup
- Resolved
-
SPARK-30091 Document mergeSchema option directly in the Python Parquet APIs
- Resolved
-
SPARK-30113 Document mergeSchema option in Python Orc APIs
- Resolved
- links to